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INTRODUCTION 


A comprehensive review and evaluation of the Langley Research Center’s 
scientific and technical information (STI) program was conducted. The purpose 
of the review and evaluation was to determine the extent to which the program 
was meeting the needs of Langley research personnel and the recipients Of 
Langley-generated STI, the areas of the program which needed improvements, and 
ways in which the program could be modified to improve its overall efficiency 
and effectiveness. The goal of the review and evaluation project was to deter- 
mine if the dissemination of the Center’s research output could be made more 
effective. 

The project utilized both survey research and systems analysis techniques. 

A steering committee composed of one representative from each research division 
was used to develop the objectives and guide the project through its completion. 
The individual tasks required to accomplish the objectives were established and 
included as phases in the project plan which is the Appendix of this report. The 
results of Phase V - Design and Evaluation of STI Systems: A Selected, Annotated 

Bibliography are contained in this report . 

STATEMENT OF THE PROBLEM 

During the 63-year history of the Langley Research Center, a comprehensive 
review and evaluation of the Center's STI program had never been conducted. 
Portions of the Langley STI program had received periodic or occasional assess- 
ment; however, no valid empirical data existed which could be used to evaluate 
the overall program. 

Purpose of the Study 

The purpose of Phase V was to compile a selected, annotated bibliography of 
literature citations related to the design and evaluation of scientific and 
technical information (STI) systems. Phase V required the use of manual and 
machine-readable literature searches; the review of numerous books, periodicals, 
reports, and papers; and the selection and annotation of literature citations. 

The results of Phase V provided a theoretical understanding and base upon which 
the methodology of the review and evaluation project was founded. 



Importance of the Study 

Phase V resulted in the compilation of a selected, annotated bibliography 
of source material devoted to the design and evaluation of scientific and tech- 
nical information (STI) systems. The bibliography was produced because (1) the • 

information was needed to develop the methodology for the review and evaluation 
project, and (2) a survey of the literature did not reveal the existence of a ' 

single published source of information pertinent to the subject. Thus, the 
bibliography represents the first or the most recent collection of information 
specifically devoted to the design and evaluation of STI systems. 

Scope of the Study 

The study was limited to (1) searches of manual and machine-readable data 
bases; (2) bibliographies pertaining to the evaluation of information systems; 

(3) books, periodicals, reports, annual reviews, conference proceedings, and 
research specifically concerned with information systems; and (4) bibliographies 
and references found in a variety of published sources. In compiling the bibli- 
ography, emphasis was placed on those references published in the time period 
between 1974 and 1980. However, certain references dated prior to this period 
were included because of their significance. 

GLOSSARY 

AGARD 
ARIST 
AS IS 
As lib 
CDA 
COSATI 
DDC 
DoD 
DSA 
OTIC 
ERIC 
IEEE 
JASIS 
LaRC 


Advisory Group for Aeronautical Research and Development 

Annual Review of Information Science and Technology 

American Society for Information Science 

Association of Special Libraries 

Comprehensive Dissertation Abstracts 

Committee on Scientific and Technical Information 

Defense Documentation Center 

Department of Defense 

Defense Supply Agency 

Defense Technical Information Center 

Educational Resources Information Center 

Institute of Electrical and Electronic Engineers 

Journal of the American Society for Information Science 

Langley Research Center 
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LISA 

MEDLARS 

MEDLINE 

M.I.T. 

NASA 

NSF 

NSTIF 

NTIS 

RECON 

SCAT! 

SSIE 

STI 

UNESCO 


Library and Infoirmation Science Abstracts 
Medical Literature Analysis and Retrieval System 
MEDURS On-Line 

Massachusetts Institute of Technology 
National Aeronautics and Space Administration 
National Science Foundation 

NASA Scientific and Technical Information Facility 
National Technical Information Service 
Remote Console Interactive Computer System 
Scientific Communication and Technology Transfer System 
Smithsonian Science Information Exchange 
Scientific and Technical Information 

United Nations Educational, Scientific, and Cultural Organization 


CONDUCT OF THE STUDY 

Phase V required the compilation of a selected, annotated bibliography on 
the design and evaluation of STI systems. The principal purpose of the study 
was to provide a single source of information pertinent to the subject. This 
information was used in the development of the methodology for the review and 
evaluation project and could also be used by others interested in this aspect 
of information science. 


Design 

The scope of this bibliography focused on literature pertaining to the 
design and evaluation of STI systems. Citations covering the general topic of 
information and such topics as the design and evaluation of information systems, 
information use and needs, the economics of information, and information pro- 
ducts and services were included because they contained substantial or innovative 
material pertinent to the subject. 

Several bibliographies were discovered during the search of the literature 
and were consulted during the preparation of this bibliography. The last com- 
prehensive bibliography on the evaluation of information systems, which was 
compiled by Beth Krevitt and Belver Griffith and published in 1973, represented 
an updating of the bibliography on the same topic which was compiled by Madeline 
Henderson and published in 1967. In 1978, Donald W. King published a compilation 
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of key papers on the design and evaluation of information systems. Although 
not a bibliography, the work by King does provide a comprehensive set of papers 
that covers a range of topics pertinent to the subject. 

Daniel N. Woods compiled and published in 1971 a bibliography on user 
studies which was devoted to the information gathering habits of engineers 
and scientists. Sylvia Faibisoff compiled and published in 1976 a comprehensive 
review of the literature on user studies which included citations devoted to 
the Information needs among scientists and engineers. Since 1977, Mary E. Young 
has published a continuing bibliography on user needs in documentation and 
information. The Young bibliography covers a variety of information needs, 
including those of engineers and scientists. 

In 1971, Harold Anker Olsen published a comprehensive bibliography on the 
economics of information. In 1972, a supplementary listing was published. 

Both editions contained citations which embrace the importance of economic con- 
sideration in information services, including information systems. In 1980, 
Donald W. King, Nancy K. Roderer, and Harold A. Olsen published a compilation 
of key papers on the economics of information. While not a bibliography, their 
work does represent the most current thinking concerning the costs, budgeting, 
and financial operation of informations systems. 

Citations in these bibliographies pertinent to the design and evaluation 
of STI systems were included in this bibliography. Preparation of this bibli- 
ography began in February 1980, thus excluding publications from late 1980. 

The cutoff was necessary to meet the timetable established for the project. 
Approximately 200 citations have been included. 

Methodology 

The basic tools used to compile this bibliography were (1) searches of 
machine-readable data bases via CDA, ERIC, LISA, NTIS, NASA RECON, and SSIE; 

(2) Library Literature and ISA; and (3) ASIS conference proceedings. Extensive 
use was' also made of bibliographies in the ARIST. The bibliographies and ref- 
erences of the citations contained in this publication were also used as sources 
of possible citations. 



Availab il ity 

Every attempt was made to Identify the availability of the citations 
Included in this bibliography. Citations which were not published or readily 
available were excluded. Works by the same author (s) having similar titles 
were included in an attempt to increase the user's ability to obtain the desired 
material. When both PB and ED numbers appeared for the same citation, both 
were included . 

Citations which appeared as NTIS documents can be purchased from the 
National Technical Information Service, 5285 Port Royal Road, Springfield, 

Virginia 22161. Citations which appeared as ERIC documents can be purchased 
from the ERIC Document Reproduction Service, P.O. Box 190, Arlington, Virginia 
22210. 

Classification 

Four subject areas were used to classify the selected citations. These 
areas included (1) information - general; (2) information systems - design and 
evaluation including information products and services; (3) information - use 
and need; and (4) information - economics. Each citation was listed under one 
and only one area within the classification scheme. 

RELATED RESEARCH AND LITERATURE 

The development of the methodology and procedures for the review and 
evaluation of the Langley scientific and technical information program which 
would provide maximum effectiveness required a complete understanding of (1) the 
operational dimensions of the Langley STI program and (2) the theoretical basis, 
Including relevant research, for the design and evaluation of STI systems. The 
operational understanding was acquired from the familiarity and experience with 
the Langley STI program. The theoretical basis, which was needed to determine 
the methods and concepts essential to design and evaluation, was developed from a 
thorough understanding of the related research and literature. An historical over 
view of the evolution of the evaluation process, which reflected the changes in 
attitudes, emphasis, techniques, and technological developments in the design and 
evaluation of information systems, was prepared and presented chronologically. 
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The User Dimension 


Prior to the 1950' s, few studies of the users' needs had been performed 
although information workers had been providing a variety of information products 
and services for years. The first user studies, which were conducted during the 
late 1950's and early 1960's, involved engineers and scientists as subjects. 

Most of these studies were conducted on a small scale, with participants drawn 
from a specific discipline, the broad population of engineers and scientists, 
or, less frequently, from users of specific systems. The technique most 
commonly employed to elicit knowledge concerning the general information- 
gathering habits and need* of users was the self -administered questionnaire. 

The results of these studies revealed that the information needs and habits of 
engineers and scientists were more diverse than had previously been thought, 
thus dlsmlnishing the potential use of the results in terms of the design and 
evaluation of information systems. 

The Flow of Scientific and Technical Information 

One of the earliest methodological studies was conducted by Menzel (1) . 
Menzel reviewed papers published prior to 1960 on the flow of information among 
scientists. He found only 26 studies which contained empirical findings. 
Techniques cited by Menzel from these studies included; examination of library 
circulation records, analysis of reference questions, self-observations , 
observations by others, diaries, questionnaires, interviews, and reference 
counts. Paisley (2) reviewed the literature on the flow of scientific litera- 
ture up to December 1965 and found only 33 studies which used empirical research 
to document information-gathering activity. 

Information Needs/User Studies 

By the end of 1963, empirical research on the information needs and uses 
of scientists and technologists appeared in the literature. In addition to 
preference, demand, and experimental studies, research during the mid-1960's 
began to take other directions. During this time, investigations were conducted 
concerning the habits and patterns of scientists in terms of information 
gathering, use, and dissemination. These investigations focused on the infor- 
mation channel(s) to determine the flow of information within the scientific 
community . 



The critical incident study appeared at this time. The critical Incident 
study recorded sample information - receiving episodes i including a description 
of the event, the communication function this event served, and the scientist's 
satisfaction (or lack thereof) with the information obtained. The studies by 
Rosenbloom (3) and the Auerbach Corporation (4) illustrated the critical 
Incident methodology. A further refinement of this technique was the critical 
Incident decision study which concentrated on information-receiving events that 
caused decisions in the work of scientists. M.I.T.'s Research Program on the 
Management of Science and Technology (5) produced a number of diverse and 
creative approaches to information transfer studies. One approach ptudled the 
communication channels used during different stages of completion of research 
projects. Another was the "matched case approach" which employed the same 
research design to compare the performance of the two or more groups being 
studied. 

In commenting on the literature of 1966 dealing with user studies, Herner 
and Herner (6) reviewed papers which utilized the following techniques; user 
records, personal interviews, observation, and questionnaires. Herner and 
Herner noted the following seven problems related to the 1966 user studies; 
the few techniques used, the diversity of user groups to which these few 
techniques were applied, the diversity and ambiguity of the language used to 
discuss the techniques and their results, the absence of innovation, the failure 
to profit from prior mistakes, and the absence of rigorous experimental design. 

Evaluation of Information Systems and Services 

The problems inherent to the design and evaluation of information systems 
and services received increased attention in the literature and were discussed 
at national and international meetings. At the 1966 annual convention of the 
American Documentation Institute, a panel focused on a number of basic issues 
including the problems in methodology, relevance as a measurement criterion, 
replication of experiments, and difficulties in making generalizations based 
on past research. Persistent methodology and conceptualization problems, 
together with a scarcity of empirical data, plagued attempts to discuss the 
actual performance of Information systems and services. 
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Snyder (7) published a critical review of the design and evaluation 
literature and identified the following methodological problems; confounding • 
of independent variables; using only retrieval effectiveness criteria to 
examine the effects of Independent variables, using gross measures to determine 
relevance, insufficient control of possibly significant sources of variance, 
Inadequate sampling procedures, Inadequate statistical techniques, failing to 
discuss other relevant research results, and failing to report sufficient 
methodology details. Marks (8) reached the same general conclusions as Snyder 
after reviewing behavioral research in information retrieval. 

Salton (9) reported on the continued development of a specialized 
computerized information and retrieval system. A large number of automatic 
retrieval procedures were tested. Including automatic information analysis, 
automatic dictionary construction, and iterative search procedures with user 
interaction. The final report of the Aslib-Cranfield Project (10), issued in 
1966, represented an extensive and complex effort in analyzing the effect of 
index language devices on recall and performance of an information retrieval 
system. Considerable debate appeared in the literature concerning the merits 
and shortcomings of this work. 

Systems Approach to Design and Evaluation 

The literature of 1967 showed a trend toward the systems approach to design 
and evaluation of information systems, as reflected in the attention given to 
inforttiation components and their interactions. A discussion of the pros and 
cons of recall and precision as measures of information system performance 
occurred during this period. The necessity of cost-benefit analysis of 
competing systems was recorded. Operations research, a general analytical and 
evaluation method, was widely used in the evaluation of Information systems 
in the late 1960's. 

Trends toward automation were evident in the literature of 1968. Publi- 
cations reflected a maturing discipline, with documented accounts of actual 
steps toward automation Including cost analyses. Man/machine interfaces in 
projected on-line computerized retrieval systems began to receive attention 
in the literature, but a recognition was growing that system evaluation 
measures needed to be broader than mere retrieval performance. 



Informal and Formal Information Seeking 

By 1968, the number of studies concerned with social and behavioral 
aspects of information-seeking had increased substantially. In addition 
many user studies employed more sophisticated techniques to elucidate particular 
characteristics of user behavior. Citation analyses and indirect observation 
techniques were introduced. Informal information transfer began to be studied 
and recognized as a major channel of information flow. The concepts of "gate- 
keepers" and "invisible colleges" appeared. Although information acquisition 
and use became more fully understood during this period, the increased know- 
ledge was not directly applied in system design. Computers became more widely 
used in information systems. 

New Trends; National Information Systems and Networking 

Considerable activity in the design of comprehensive national information 
systems devoted to broad subject disciplines occurred during 1969. In the U.S., 
most programs were designed and implemented by leading scientific and technical 
societies such as those lA the fields of physics, mathematics, and linguistics. 
The programs consisted of a machine-readable data base of indexed citations 
from which a variety of bibliographic products could be generated, including 
indexes, literature searches, bibliographies, and selective dissemination 
services. An example of such a system was NASA's RECON. 

Another important trend around 1969 was the Integration and connection of 
bibliographic services by networking concepts. Some of the first such networks 
were MEDLARS and ERIC. By this time, only a small number of actual evaluations 
of operating systems had been done. MEDLARS was the only large national infor- 
mation system which had been thoroughly evaluated. The availability of on-line 
systems revived interest in research on automatic indexing, classification, 
vocabulary control, and automatic searching. Concept definition, methods of 
data collection, and theory had developed so that now there was some con- 
sistency in observations about user needs and uses. Recognition of the growing 
importance of cost effectiveness was reflected in the literature of 1969. 
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Economic Consideration 


By 1970, economic restrictions had become the most important factor in the 
design, operation, and evaluation of information systems. Managers of existing 
systems were called upon to justify costs, and proposals for new systems were 
carefully studied and compared with the Informal sources used by researchers. 
In-house testing of existing systems was growing, based on the realization 
that the support and respect of users, potential users, and management were 
essential. Cost effectiveness as applied to operational decision-making was 
the focus of the work of a group of researchers and Information specialists, 
notably Lancaster, King, and Bryant. Research by these individuals led to 
emphasis on evaluating the cost effectiveness and benefits tradeoff of infor- 
mation systems . Evaluation began to impact more importantly on specific deci- 
sions concerning system design, implementation, modification, and discontinuance. 

Aspects of Needs and Uses 

Studies concerned with Information needs and uses on national and inter- 
national basis emerged in 1971. Systematic studies were conducted in the 
U. S. at Stanford University, Johns Hopkins, and by the American Psychological' 
Association. The 1970* s produced many studies of the information needs and 
uses of scientists and engineers in other countries, including Russia, East 
Germany, England, Canada, and the Netherlands. International cooperation 
among scientists through informal communication networks was a topic represented 
in the papers of the early 1970* s. 

Research which led to the understanding of the social structure and 
organization of the research environment was undertaken during this period. 

Work performed by Thomas J. Allen at M.I.T (11) determined the structural 
characteristics of R&D laboratories. Informal relationships and physical 
locations were shown to have direct bearing on information transfer within 
an R&D organization. Rubenstein investigated the communication systems used 
by engineers and scientists. The conclusions drawn from this research indicated 
that engineers tended to prefer oral communication sources, . while basic sci- 
entists relied on both oral and written channels. 
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Evolution of Evaluation Procedures 

In the literature of 1973-1974, evaluation procedures and tools tailored 
to Information system environments emerged. Debons and Montgomery (12) 
enumerated the following procedures: systems analysis, operations research, 

benefit-cost analysis, plannlng-prograramlng-budgetlng, value/analysis engineer- 
ing, management audit, and evaluation. Greater cost conciousness and more 
attention to users and user studies In nonsclentlfic disciplines were con- 
tinuing trends In information science. User satisfaction was suggested as a 
substitute for relevance in evaluating system effectiveness. 

Extended Scope of User Studies 

Crawford (13) showed that of the 95 user studies published between 1975 
and 1977, 24 were focused on physical sciences (still the largest representation); 
however, 20 were focused on the social sciences (second largest representation). 
The other categories and counts were biological/health sciences, 9; human needs, 
10; mass media, 3; general information centers/systems, 13; specific techniques 
(mostly user interfaces with on-line services), 10; and theory, 6. Of the 24 
papers in the physical science category, 10 were case presentations on the 
use of a specific system, 9 were surveys, and 5 were conceptual or review 
papers. 

A National Forum on Scientific and Technical Information was held during 
1976-1977 consisting of 375 professional users of information, providers of 
information, and those who make public policy. Five problem areas were defined, 
and it was concluded that a focal point should be established within the 
government to address these problem areas, providing both leadership and 
financial support. 

SUMMARY 

The design and evaluation of information systems had progressed from 
simplistic and unscientific research to the current sophisticated and empirical 
framework. Both the methodology (how to measure) and concepts (what to measure 
and why) differed in scope, depth, criteria, and techniques in accordance with 
their appropriateness to the evaluative purpose. The literature revealed that 
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research existed to assist In the choice of valid, reliable, reproducible 
evaluative procedures and the formulation of valid assessment measures. The 
assessment of Information systems had grown to include the organization, 
functions, and operations of the system; the products and services of the * 

system; and the clientele and organizations served by the system. A periodic 
evaluation which incorporated a combination of social science concepts and ' 

management tools and techniques was established as an essential function of 
management . 

CONCLUSIONS 

The purpose of Phase V was to compile a selected, annotated bibliography 
on the design and evaluation of STI systems. The results of Phase V provided 
a theoretical understanding and base upon which the methodology of the review 
and evaluation project was founded. 

Periodic review and evaluation of an information system should be con- 
sidered an essential responsibility of management. Evaluation should be 
(1) summative, providing decision-makers with information about the effective- 
ness of the system; (2) formative, producing information that is fed back into 
the program for Improvement during development or operation; and (3) on-going, 
disclosing the strengths and weaknesses of the system, suggesting ways to 
Improve the overall performance of the system, and providing future orientation 
and direction for system development. Evaluation of an information system 
should include the structure, functions, and operations of the system; the 
products and services of the system; and the individuals and organizations 
served by the system. However, the most meaningful parameter in the design 
and evaluation of an Information system involves the needs of the information 
user. Consequently, the needs of the Information user must be viewed as an 
essential aspect of any information system evaluation. 

The development of the Langley STI review and evaluation plan was based * 

upon these considerations. The individual tasks required to accomplish the 
objectives of the project were established and included as phases in the 
project plan which is the Appendix of this report. 
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evaluating of courses. Sample course announcements, application forms, activities 
and time tables, participant evaluation reports, budget/financial reports, and 
checklists were appended. 

Bagikian, Ben H. The Information Machines — Their Impact on Men and the Media . 
New York; Harper and Row, 1971. 

Speculations were offered on what new technical systems (computers, innova- 
tions in the transmission of signals, and new ways to feed images into and out 
of this system) would do to the content and form of news in the U.S. during the 
remainder of this century. This book focused on what the content of daily infor- 
mation would be, in what form it would be delivered, and how it would be distrib- 
uted throughout the population. It considered what technologies were likely 
to change the way the next generation received its news, what changes the rapid 
reporting of daily events would make in human affairs, what the audience for 
news in the U.S. was like, and what some of the peculiarities of the news in 
this country were. 
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Becker , Joseph . A National Approach to Scientific and Technical Information 
in the United States . Los Angeles, Calif.: Becker and Hayes, Inc., July 

1976 (Prepared for National Science Foundation; Avail. NTIS PB 261-270 
and ERIC ED 1 29-240) . 

The study articulated the federal government's responsibility in providing 
for the dissemination of scientific and technical information. It presented 
an historical overview of the field; it described the new directions of science 
today which posed obvious challenges to the scientific communication system; it 
identified and explained the pressures which were affecting the nation's ability 
to fully utilize its scientific knowledge; and it described the action the fed- 
eral government could take now to ensure the harmonious and continuing develop- 
ment of the nation's scientific communication enterprise. Based on the recom- 
mendations of past studies and reports and a discussion of the new directions 
of science, it suggested that the federal government establish (1 ) a locus 
of responsibility for making science information policy at the national level; 

(2) a dynamic, federally funded research and development program; and (3) a vol- 
untary organizational mechanism for coordination of STI acitvities in the public 
and private sectors. 

Buryi-Shmaryan, 0. E. The Principle Channels for Transmitting Scientific and 
Technical Information and Their Effectiveness Based on Survey Results . 
Boston Spa, England: National Lending Library for Science and Technology, 

1971 (Avail. NSTIP 71N-29116). 

After analysis of the operating experience of information services in a 
number of businesses, review of data from literature sources, and discussion 
with specialists, the author defined and ranked four main information channels. 
Listed in descending order of importance, they were (1 ) reference books and 
information obtained directly by the receiver of the information; (2) partici- 
pation in conferences, seminars, meetings, and exhibitions; (3) participation 
in scientific visits to firms in the particular subject interest; and (4) per- 
sonal contact between specialists while carrying out work. Relative costs for 
these channels were also discussed. 

Capital Systems Group, Inc. The Growth of Scientific and Technical Information: 
A Challenge . Rockville, Md.: Capital Systems Group, Inc., January 1974 

(Prepared for National Science Foundation; Avail. NTIS N74-22601 ) . 

This book was designed to alert editors, publishers, and others responsible 
for the primary dissemination of scientific and technical information (STI) to 
ideas, procedures, systems, and technologies that could be used to disseminate 
information more effectively and economically. It described innovations and 
improvements in several areas: innovations in conventional journal and mono- 

graphic publishing, print-on-paper alternatives to conventional publication, 
non-print paper and mixed-media innovations, trends and prospects, and innova- 
tions needed. 
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Capital Systems Groups Inc. Improving the Dissemination of Scientific and 
Technical Information: A Practitioner's Guide to Innovation. (Final 

Report ) . Washington, D.C.: Capital Systems Group, Inc., April 1975 

(Prepared for National Science Foundation; Avail. ERIC ED 118-151). 

The aim of this guide was to alert persons with an operational interest 
in scientific communication to new ideas, techniques, and equipment in the 
field of communication media and publications. The focus was on the dissemina- 
tion of scientific information via the technical journal or its equivalent. 
Secondary dissemination of information such as bibliographies, data bases, and 
services are treated incidentally. The guide was organized into five sections: 
innovations in conventional journal and monography publishing, print-on-paper 
alternatives, non-print-on-paper and mixed media innovations, trends and pros- 
pects, and innovations needed. Each section was subdivided into individual 
entries describing particular processes or Innovations. This guide was designed 
as an "idea book" to help small scientific societies, journals, and publishers 
as well as to increase general awareness of innovations and stimulate develop- 
ment of new ideas. 

Chambaud, S. C. Requirements in Scientific and Technical Information: Govern - 

ment Viewpoint . Advisory Group for Aerospace Research and Development. 

The Hague, Netherlands: Ministry of Economic Affairs, January 1979 (Avail. 

NTIS N79-2091 4) . 

Although the needs of industries for scientific and technical information 
varied according to the type of enterprise considered, it was shown that govern- 
ments could analyze and attempt to meet these needs. Three essential factors 
which influenced the way in which an industry perceived its need for information 
were the type of industry, its domain of activity, and its status (growing, 
declining, or stationary) . Industries needed information concerning research 
and production methods, technical-juridical information covering patents and 
licensing, and technical-economic data related to commerce and marketing. 

Collons, Rodger D. Incentives for the Generation, Dissemination and Use of 

Scientific and Technical Information . Philadelphia, Pa.: Drexel Univers- 

ity, September 1977 (Prepared for National Science Foundation; Avail. NTIS 
PB 276-180) . 

This report detailed a conference in which key decision-makers and special- 
ists in scientific and technical information systems were brought together with 
incentive area experts from law, economics, sociology, and psychology to address 
the issue of incentives for the generation, dissemination, and use of scientific 
and technical information. Included were workgroup reports and presentations by 
speakers. It was generally concluded that the issue of incentives was too com- 
plex to permit the developnent in any single workshop of a set of incentives 
which were particularly applicable to information systems; however, the confer- 
ence did develop insights into how to approach the issue of incentives. 


Conmittee on Scientific and Technical Information. The Role of the Technical 
Report in Scientific and Technological Communication . Washington, D.C.: 
Federal Council for Science and Technology, December 1968 (Avail. NTIS 
PB 1 80-944) . 

A study was conducted by COSATl to appraise the role of the technical 
report in the scientific-engineering communication process. Topics addressed 
included scientific journals; secondary literature (abstracts, indexes, and 
literature reviews); technical reports; reviews and monographs; economics of 
publication, informal information exchange; and the quality of technical liter- 
ature. It was concluded that technical requirements necessitated continued 
coexistence of the various scientific and technical communication media; how- 
ever, the strengths of each medium must be recognized and nurtured. 

Creager, William A., and others. Plans and Recommendations for Establishing 
and Operating a Clearinghouse for Innovation in Scientific Communication . 
Rockville, Md. : Capital Systems Group, Inc., March 1975 (Prepared for 

National Science Foundation; Avail. NTIS PB 243-461). 

The fundamental recommendation was for the establishment of a Clearinghouse 
for Innovation in Scientific Communication. It was recommended that the Clear- 
inghouse's first activity be the publication of a guidebook that would describe 
innovations applicable to primary dissemination of scientific and technical 
information. Recommendations and tentative plans were also presented for the 
gradual enlargement of the Clearinghouse's scope to include, within the first 
year of operation, the secondary and tertiary dissemination of information. 

Creager, William A., and others. Developing a Planning Guide for Innovation in 
The Diss emination of Scientific and Technical Information; Final Report of 
Phase_I. Rockville, Md.: Capital Systems Group, Inc., March 1975 (Pre- 

pared for National Science Foundation; Avail. NTIS PB 243-469) . 

This report summarized the experience of a project designed to (1) iden- 
tify and describe innovations that might be applied to the primary, recorded 
dissemination of scientific and technical information, and (2) organize the pro- 
ject's findings into a guidebook that might be used by editors, publishers, and 
others responsible for communication programs. In addition to this report, the 
project culminated in a prototype of the guidebook and a proposed plan for large- 
scale production and distribution of the guide. The final report included chap- 
ters on project history and methodology, the results of the project, and obser- 
vations concerning innovation in scientific communication. 
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Dewhirst/ H. Dudley. "Influence of Perceived Information-Sharing Norms on Com- 
munication Channel Utilization." Academy of Management Journal # X1V:3 
(September 1 971 ) , 305-31 5. 

Research on communication channel use had shown that Interpersonal channels 
were generally more efficient than written channels. However, using interper- 
sonal channels involved a psychological cost, particularly for individuals who 
believed that free and open sharing of information was not the norm. This study 
explored channel use among a group of scientists and engineers and suggested 
that channel use, both within and outside the organization, was influenced by 
the individuals' perceptions of organizational information-sharing norms. 

Fenichel, Carol, ed. Changing Patterns in Information Retrieval. Proceedings 
of the Tenth Annual National Information Retrieval Colloquium . Washington, 
D.C.: American Society for Information Science. 1974. 

The theme for the papers presented at the Tenth Annual Information 
Retrieval Colloquium was the state-of-the-art of information science. The 
decennial colloquium attempted to update and extend the symposium presented at 
the 1961 Annual Meeting of the American Documentation Institute (ADI). The 
papers reviewed the growth and change within the field and predicted future 
directions for information science. 

Ganz, Carole. "The Role of Scientific Communication in the Process of Techno- 
logical Innovation." Aslib Proceedings , XXVIII (November/December 1976), 
385-91 . 

The results of two projects aimed at increasing the understanding of the 
role and contribution of scientific and technical information the process of 
innovation were summarized. One project concentrated primarily on the kinds 
and channels of scientific and technical information used by the innovators 
during the ideation/conception stage. The second project provided a framework 
for understanding the flow of scientific and technical information in applied 
research. Both studies concluded that people, and not formal channels of com- 
munication, were more effective for transmitting technical information. In addi- 
tion, frequency of communication among researchers was found to be determined 
more by the architecture of the work place than by the technical needs of the 
research group. 

Gellman Research Associates, Inc. Social Control of the Scientific and Techni- 
cal Information System; Public Utility Regulation and International Trans- 
mission (Final Report) . Washington, D.C.: Gellman Research Associates, 

January 1977 (Prepared for National Science Foundation; Avail. NTIS 
PB 279-425) . 

The scientific and technical information (STI) system which supports busi- 
ness, government, and society was studied. The study concerned government 
management of STI flow and the application of public utility regulation. The 
focus was on the future applications of technology and better uses of resources 
in STI. Also considered were (1) STI and its relationship to society, (2) STI 
regulation, (3) adaptations in STI, (4) forecast of STI technologies, (5) STI 
in foreign affairs, and (6) STI as a regulated public utility. 
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General Accounting Office. Better Information Management Policies Needed: A 

Study of Scientific and Technical Bibliographic Services . Washington/ 

D.C.: General Accounting Office/ August 1979 (Avail. GAO PSAD-79-62) . 

GAO'S study of government information centers providing bibliographic ser- 
vices to the scientific and technical community was presented. It confirmed 
the need for better management. Evidence of duplication/ proliferation of 
facilities/ and inconsistent cost recovery practices were found. The vagueness 
of authorizing laws and function statements was found to contribute to the 
duplication of services. It was recommended that each department and agency 
designate a top official to coordinate and manage its information/ and that the 
Office of Management and Budget establish a committee to coordinate government 
scientific and technical Information activities. 

The George Washington University Medical Center. Preliminary Report of the 

Providers' Perspective Conference of the National Forum on Scientific and 
Technical Conununi cation . Washington/ D.C.: National Science Foundation/ 

February 1 977 (Avail. NTIS PB 269-058) . 

Problems and issues encountered in adequately meeting the needs of scien- 
tific and technical information users were addressed. The viewpoints and per- 
ceptions of professional librarians/ publishers/ information specialists/ and 
educators were reported. 

The George Washington University Medical Center. Preliminary Report of the 
Policymakers' Perspective Conference of the National Forum on Scientific 
and Technical Communication . Washington/ D.C.: National Science Founda- 

tion, August 1977 (Avail. NTIS PB 272-988). 

The conference addressed the question, "Are there current needs for public 
policies affecting scientific and technical communication in the United States, 
and if so, what are the options?" Policymakers and their staffs, experts in 
scientific and technical communication, and decisionmakers from all sectors came 
together to discuss their viewpoints and perceptions about the problems and 
issues related to scientific and technical information. 

Giuliano, V., and others. Passing the Threshold into the Information Age — 

Perspective for Federal Action on Information. Volume I — Basic Findings 
Report . Cambridge, Hass.: Arthur D. Little, Inc., January 1978 (Prepared 

for National Science Foundation; Avail. NTIS PB 281-720). 

Scientific and technical information transfer has had only low priority for 
national attention recently. Three eras of scientific information transfer were 
identified: a continuing traditional era that emphasized transfer of disciplin- 

ary knowledge, a post World War II era devoted to mission-related information, 
and an emerging era oriented toward socio-economic-technical systems problems. 

A description of the information eras and their implications, current issues 
and problems, the stakeholders involved and their perspectives, and possible 
action steps to improve the present infrastructure were provided. 





Griffithr Belver C., ed. Key Papers in Information Science . White PlainSf 
N.Y.: Knowledge Industry Publications, Inc., 1980. 

More than thirty key papers from a variety of interdisciplinary academic 
and professional journals and publications were presented in this volume. The 
contents were divided into the following five subject areas: Structure and 

Dynamics of Science Information Plow, Innovation: Required Plow of Knowledge, 

The Structure of Literature and Documents, Information Retrieval and Analysis, 
and Tools and Ideas. 

Hess, Edward J. "Toward the Measurement of Benefits Prcan the Use of Information 
Services." Innovative Developments in Information Systems: Their Benefits 
and Costs. Proceedings of the 36th Annual Meeting of the American Society 
for Information Science . Edited by Helen J. Waldron and F. Raymond Long. 
Washington, D.C.: American Society for Information Science, 1973, 91. 

This paper proposed a dependable and systematic feedback loop for monitor- 
ing the use of media by clients and staff members to determine who benefited 
from the media, and to some extent what benefits were derived. Purposes for 
which the media were likely to be used were influenced by the environment in 
which the information system might find typical purposes for its use to be 
(1) support of the instructional program, (2) research, or (3) general trans- 
mission of the cultural heritage not specifically related to the curriculum. 

Johnson, J. H., and Ivy, L. Development of a Science and Technology Information 
System . Oklahoma City, Okla.: Oklahoma State Legislative Council, March 

1979 (Avail. NTIS N79-34091 ) . 

The activities of the Oklahoma Legislative Council to establish capabili- 
ties in the area of systematic transfer of scientific and technical Information 
were described. Oklahoma was one of 42 states involved in the State, Science, 
Engineering, and Technology (SSET) program initiated to provide state policy- 
makers with much of the best technical information possible to be used in the 
decision making process. The survey technique was used to obtain evaluations 
from a random sample of the legislature for the information resources available 
to them. Both needs and resource assessments were presented, including a series 
of models describing several types of information for the purpose of evaluation, 
a resource directory, flow charts, and an organizational chart. Questionnaires 
for needs assessment and resource assessment were contained in the appendices. 
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Johnston, Ron, and Gibbon, Michael. "Characteristics of Information Osage in 
Technological Innovation." IEEE Transactions on Engineering Management 
(February 1975), 27-34. 

The characteristics of information which contributed to the resolution of 
technical problems during thirty on-going Innovations in British industry were 
studied to determine the patterns of information flow consonant with successful 
innovation. Three findings emerged: (1 ) Information obtained from literature 

contributed as much as that from personal contact. The innovation process was 
most efficient when these two sources were used in a mutually supportive fash- 
ion. (2) Different sources were selectively used to obtain different types of 
information. The choice of appropriate sources in an appropriate order had a 
considerable impact on the innovation process. Improvement in the choice pat- 
tern was associated with better educated problem-solvers and research-intensive 
firms. (3) No direct correlation was found between the frequency of use of a 
source and its relative contribution to the innovation. Some vital pieces of 
information were obtained from sources infrequently used. Implications for 
researchers and management were drawn. 

King, Donald W., and others. Statistical Indicators of Scientific and Technical 
Communication (1960-1980). 1977 Edition . Rockville, Md.: King Research, 

Inc., May 1 977 (Prepared for National Science Foundation; Avail. NTIS 
PB 278-279) . 

Research in developing statistical indicators of scientific and technical 
communication was described, with special emphasis on the periodical literature 
including new estimates of the number of journals and other periodicals, charac- 
teristics of journals, journal prices, number of subscribers, type of publishers, 
and coverage by abstracting and listing services. Special analyses were also 
presented for the value of the journals and for the economics of page charges. 

King, Donald W. , and others. Statistical Indicators of Scientific and Technical 
Communication (1960-1980). Volume I: A Summary Report . Rockville, Md.: 

King Research, Inc., Wlay 1976 (Prepared for National Science Foundation; 
Avail. NTIS PB 260-374) . 

Statistical indicators of scientific and technical communication were pro- 
vided for use by planning and policy decision makers and managers of scientific 
and technical information services. An analysis of past trends and projections 
to 1980 were included. 

King, Donald W., and others. Statistical Indicators of Scientific and Technical 
Ccmmunication (1960-1980). Volume II: A Research Report . Rockville, Md.: 

King Research, Inc., May 1976 (Prepared for National Science Foundation; 
Avail. NTIS PB 254-060). 

The data, framework, and mathematical models used to generate the statisti- 
cal indicators for the formal scientific and technical literature, such as 
books, journal articles, and technical reports, were described. A bibliography 
and appendices describing the methodology were included. 


23 


King, Donald W. , and others. Statistical Indicators of Scientific and Technical 
Communication (1960-1980). Volume III; A Data Appendix to Volume II . 
Rockville, Md.; King Research, Inc., May 1976 (Prepared for National 
Science Foundation; Avail. NTIS PB 255-503) . 

Data from a longitudinal study of 1 91 journals published between 1 962 and 
1 974 were analyzed. Tables showed characteristics of these journals, such as 
number of issues, number of articles, number of pages, number of characters per 
page, and bibliographic citations. The work was presented as a data reference 
tool to other researchers involved in the study of the scientific and technical 
journal literature. 

King, Donald W. , and others. The Journal in Scientific Communication; The 
Roles of Authors, Publishers, Libraries and Readers in a Vital System . 
Rockville, Md.; King Research, Inc., May 1979 (Prepared for National 
Science Foundation; Avail. NTIS PB 296-263) . 

The scientific and technical journal system in the United States was 
described. Data about authorship, publishing, libraries, abstracting and index- 
ing, and reading of scientific and technical journals were included. The flow 
of information among the principal participants in this system was also dis- 
cussed. Some data were derived from surveys of authors, publishers, abstract- 
ing and indexing services, emd scientists and engineers (as readers) . Substan- 
tial data were given on the cost of the entire system. Much of the information 
dated back to 1960 and was projected to 1985. 

King, Donald W. , and Roderer, N. K. Systems Analysis of Scientific and Techni- 
cal Communication in the United States. Annex 1; Communication Functions 
in Science and Technology . Rockville, Md. : King Research, Inc., May 1978 

(Prepared for National Science Foundation; Avail. NTIS PB 281-848). 

The development of a functional framework for scientific and technical 
journal communication was documented. Generic functions, origination, transmis- 
sion, recording, preservation, and end-use were defined. Both the current prac- 
tices and the electronic alternatives were specified in terms of activities 
comprising these functions. The bases for the functional definitions of scien- 
tific and technical coimounications were also presented. 

King, Donald W., and Roderer, N. K. Systems Analysis of Scientific and Techni- 
cal Communication in the United States. Annex 2; The Current Practice . 
Rockville, Md. ; King Research, Inc., May 1978 (Prepared for National 
Science Foundation; Avail. NTIS PB 281-849). 

The current practices in scientific and technical journal communication in 
the United States, focusing primarily on 1975, were described. Cost and volume 
data were presented for the activities of authors, publishers, libraries, secon- 
dary services, and end-users. Analyses included consideration of three sizes of 
journals, four fields of science, three communication paths, and trends observed 
over time. A major focus was the economic interactions between participants. 


Kingf Donald W. / and Roderer, N. K. Systems Analysis of Scientific and Techni - 
cal Communication in the United States. Annex 3; An Electronic Alterna~ 
tive » Rockville^ Md.: King Research, Inc., May 1978 (Prepared for 

National Science Foundation; Avail. NTIS PB 281-850). 

A likely electronic alternative to current journal communication practice 
was described in terms of generic functions. The annex discussed positive and 
negative factors which would influence developments in this area. 

King, Donald W. , and Roderer, N. K. Systems Analysis of Scientific and Techni- 
cal Communication in the United States. Annex 4; The Cost Model . 

Rockville, Md.: King Research, Inc., May 1978 (Prepared for National 

Science Foundation; Avail. NTIS PB 281-851). 

The cost model used in the systems analysis of scientific and technical 
journal communication was described. A large number of model parameters reflect- 
ing volume, process, and cost information were associated with the activities 
performed under each function. By selecting appropriate parameters, it was 
possible to simulate either current practice or the electronic alternative 
under a wide variety of input process and output conditions. The annex 
Included descriptions of model c<xnponents, methodologies for using the model, 
and sane specific model results. 

Miller, Richard H., and others. Innovation in the Dissemination of Scientific 
and Technical Information . Stanford, Calif.: Applied Communication 

Research, Inc., April 1975 (Prepared for National Science Foundation; 

Avail. NTIS PB 244-333) . 

This planning guide provided an introduction to each of the many multi- 
disciplinary procedural tools and technological innovations applicable to scien- 
tific and technical information communications, with discussion of their speci- 
fic applications. It was designed to help the reader make an informed first 
step toward choosing and implementing such innovations. The planning guide was 
organized for reader-controlled access to its information. 

Myatt, Dewitt 0., and Jover, Susan I. Compilation of Major Recommendations From 
Five Studies Relating to National Scientific and Technical Information 
Systems . McLean, Va.: Science Communication, Inc., April 1970 (Prepared 

for National Science Foundation; Avail. NTIS PB 193-345). 

One hundred twenty-five recommendations from four studies commissioned by 
OOSATI and one performed by SATCOM of NAS-NAE were included. Part I presented 
the recommendations as concise statements, listed according to the subject cate- 
gories of central management concepts for national programs, roles and responsi- 
bilities of organizations generating information for the scientific and techni- 
cal community, and suggested techniques for approaching areas such as user/ 
operator education, standardization, informal communications, and literature 
handling. Part II presented the recommendations individually, giving the con- 
cise statement form and the full text of each recommendation, page numbers of 
important related discussion in the report, other related recommendations in 
the report, and additional annotation on background and import of the recommen- 
dation which might not be immediately apparent. 
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National Science Foundation. Annotated Bibliography, 1973-1977 . Washington, 
D.C.: National Science Foundation, June 1979 (Avail. NTIS PB 298-866). 

This bibliography contained the resumes of reports, articles, conference 
papers, and other publications produced during 1 973-77 by projects supported 
by the Division of Science Information. The bibliography contained 123 cita- 
tions which were listed alphabetically by year. The citations were indexed 
by author, corporate source, title, award number, and subject. 

National Science Foundation. Federal Scientific and Technical Communication 
Activities; Annual Report, January-December 1976 . Washington, D.C.; 
National Science Foundation, 1976 (Avail. NTIS PB 272-800). 

Descriptions were given of the federal scientific and technical informa- 
tion activities of more than 60 programs during 1 976, including the Library of 
Congress and the Smithsonian Institution. The descriptions, provided volun- 
tarily by federal agencies, emphasized new developments and achievements rather 
than routine ongoing activities. A prefatory chapter analyzed and summarized 
significant activities, developments, and trends within the federal scientific 
and technical information community, based on the agencies' sulxnissions. Appen- 
dices included a glossary of acronyms, a listing of key personnel associated 
with individual programs, and a general index. 

Nelson, Carnot E., and Pollock, Donald K., eds. Communication Among Scientists 
and Engineers . Lexington, Mass.; Heath Lexington Books, 1970. 

This volume included papers presented at a conference on "Communication 
Among Scientists and Technologists" sponsored by Johns Hopkins University and 
the National Science Foundation. The objectives of the conference were: to 

bring together researchers and system developers to discuss the current models 
of the processes by which scientists and technologists communicate, to present 
system developers with research results to date, and to encourage the discussion 
of the applicability of current research to the development of science informa- 
tion programs. The volume dealt with the communication structure of science 
and the production of scientific information, the utilization of scientific 
information, and the development of scientific information systems. 

Nlsenoff, Norman, and Clayton, Audrey. A Forecast of Technology for the Scien - 
tific and Technical Information Cannmunities . Arlington, Va. : Forecasting 

International, May 1 976 (Prepared for National Science Foundation; Avail. 
NTIS PB 253-937) . 

This study was an exploration of the future of scientific and technical 
communication. Primary achievement of the project was the development for the 
first time of a unified body of data representing "best judgment" forecasts of 
communication system performance, costs, and availability for the next 25 years, 
in conjunction with a parallel of the usage patterns and needs of representative 
user groups in the scientific and technical community. 



Olson, Edwin E. Organizational Factors Affecting the Flow of Scientific and 
Technical Information in Industrial R&D Divisions: Final Report . 

Rockville, Md.s Research Division Westat, Inc., January 1978 (Avail. NTIS 
PB 277-761 ). 

Based on an organizational diagnosis of the R&D divisions of three com- 
panies, this exploratory study found that the information-seeking behavior of 
scientists and engineers emd their effectiveness in obtaining information were 
affected by the organization's work structures and processes, by the patterns 
of interpersonal interactions and organizational climate, and by the rewards 
and support they received for taking risks to seek information. For example, 
the functional organization of an R&D unit could lead to barriers between the 
information specialists and those who need technical information services. In 
companies where the climate did not support information-sharing across project 
or department lines, the informal information channels were under-utilized. 
Individuals who perceived it as risky to ask for or provide technical informa- 
tion also believed that their work groups were ineffective in getting the infor- 
mation they need. 

Olson, Edwin E., and others. The Impact of Behavioral and Technical Information 
Interventions on Industrial R&D Projects . Rockville, Md.s Capital Systems 
Group, Inc., November 1979 (Avail. NTIS PB 80-110265). 

The acquisition, use, and transfer of information both within the research 
and development lab and between the research and development lab and the engi- 
neering and manufacturing divisions in one company were investigated. The 
experiments included information searches for Individual projects. Information 
specialists joining project teams, workshops to improve the management of infor- 
mation, and Improved ccximunlcation between research and development scientists 
and engineers and their clients in the operating divisions. Findings about the 
correlation of behavioral and technological changes in technical information 
with the quality of research and development results and implications for aug- 
menting existing formal and informal channels of communication were discussed. 

Office of Science Information Service. Federal Scientific and Technical Ccanmun- 
ication Activities; 1 974 Progress Report . Washington, D.C.; National 
Science Foundation, June 1975 (Avail. NTIS PB 242-318). 

This report presented highlights of 1 974 activities of more than 50 federal 
scientific and technical information programs in 13 major departments and inde- 
pendent agencies, the Smithsonian Science Information Exchange, the Government 
Printing Office, and the Library of Congress. Each program prepared its own 
description. The Office of Science Information Service, National Science Foun- 
dation, compiled the resulting descriptions and prepared a list of acronyms and 
abbreviations and an index. 
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Rathswohl, Eugene J. "Educating Management Students in Information Utiliza- 
tion." The Value of Information. Collection of Papers Presented at the 
6th Mid-Year Meeting of the American Society for Information Science . 
Syracuse University, Syracuse, N.Y.: American Society for Information 

Science, May 1977, 140~147. 

The rationale and approach taken in designing two undergraduate management 
courses in the area of information utilization at the University of Texas were 
described. The courses were conceived as user education courses and were 
offered as electives in the undergraduate management major curriculum. They 
were designed on the premises that undergraduate management students needed to 
heighten their awareness of the possibilities provided by information resources 
and that as information users they needed to develop practical skills for cop- 
ing with the realities of existing information arrangements. 

Rosenbloom, Richard S., and Wolek, Francis W. Technology and Information 
Transfer . Boston, Mass.: Harvard University, 1970. 

By analyzing the various sources used by the researcher as functional ele- 
ments of a single system, the authors explored the nature of the information 
and the circumstances which led to the acquisition of the information. The 
corporations surveyed were engaged in the creation of new technology; however, 
the government contracting laboratory and the small technically oriented firm 
were eliminated. An analysis was offered by categories of information sources, 
by task, by experience in discipline, by degree of seniority, by educational 
level, by professional activity, and by periodical readership. Prom these 
results, a predictive model was established. The final conclusion was that the 
individual himelf provided the core for the networks of sources of information 
available to him. 

Saracevic, Tefko. "Ten Years of Relevance Experimentation - A Summary and Syn- 
thesis of Conclusions." The Information Conscious Society. Proceedings 
of the 33rd Annual Meeting of the American Society for Information Science . 
Edited by Jeanne B. North. Washington, D.C.: American Society for Infor- 

mation Science, 1970, 33-36. 

This paper described briefly some two dozen experiments concerned directly 
with relevance which have appeared in the literature over the past 10 years, 
and summarized the experimental findings, especially as they were applicable 
to the practice of information handling. It was believed that these experiments 
represented the totality of experimental work on relevance. The experiments 
were basically concerned with the effects of a variety of variables on human 
relevance judgment. The conclusions were arranged in terms of five major 
classes of variables investigated: documents and document representations; 

queries; judgmental situations and conditions; modes for expression; and 
people — the human characteristics. 
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SaraceviCf Tefko» e<3. Introduction to Information Science . New York: R. R. 

Bowker Company, 1970. 

This book compiled 66 journal articles representative of the information 
science literature published during the 1960's. Broad topics covered were basic 
phenomena, information systems, and evaluation of information systems. The 
articles demonstrated the complexity and interdisciplinary nature of informa- 
tion science. 

Shapero, Albert, and others. The Effective Use of Scientific and Technical 

Information in Industrial and Non-Profit Settings: A Study of Managerial 

Interventions . Austin, Tex.: University of Texas, June 1978 (Prepared 

for National Science Foundation; Avail. ERIC ED 165-720). 

A study conducted in a nonprofit research and development organization and 
in the technical development department of a profit corporation was designed 
to develop and implement interventions that would modify the information- 
communication behaviors of the technical professional in these organizations 
and measure and analyze the effects of the interventions. The four interven- 
tions studied included the hiring of a "high communication" — an individual 
who tended the gates of technical information flow within an organization; the 
rearrangement of offices and people within a department; the resignation of 
a "high communicator"; and the progress of individual projects through time. 
Findings suggested a need for a close look at the "high communicator" concept 
to account for the professional's individual preferences in function and style, 
as well as differences encountered over time. 

Strawhorn, John M., and Creager, William A. "Innovations to Improve the Primary 
Dissemination of the Scientific and Technical Information." Information 
Revolution: Proceedings of the 38th Annual Meeting of the American Society 

for Information Science . Edited by Charles W. Husbands and Ruth L. Tighe. 
Washington, D.C.: American Society for Information Science, October 1975, 

99-1 00. 

The results of a study of innovations in the primary dissemination of 
information through the technical journal or its equivalent were presented. 

Pour trends were identified (1) pluralism in communication - growing diver- 
sity and flexibility; (2) the foregoing of some aesthetic values for greater 
econcxny; (3) increasing exploration of ways of involving the author in editor- 
ial processing and composition; and (4) a tendency by publishers to differen- 
tiate products for institutions from those for individuals. The results of the 
study were used to prepare a guide. Improving the Dissemination of Scientific 
and Technical Information: A Practitioner ' s Guide to Innovation. 
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Strawhorn, John M. / and others. Improving the Dissemination of Scientific and 
Technical Information; A Practitioner's Guide to Innovation . Rockviller 
Md.: Capital Systems Group, Inc., April 1975 (Avail. NTIS PB 247-057). 

The Guide's focus was on the primary dissemination of scientific infor- 
mation, the technical journal or its equivalent. It dealt with the initial 
recorded transmission of information. The Guide's contents were organized into 
sections, according to the degree of departure from conventional journal and 
monographic publishing. Each section was made up of a number of individual 
entries describing particular innovations. 

U.S. Congress, Senate, Committee on Labor and Public Welfare. Federal Manage- 
ment of Scientific and Technical Information (STINFO) Activities; The Role 
of the National Science Foundation . Washington, D.C.; Government Printing 
Office, February 1976 (Avail. ERIC ED 139-373). 

An overview of the role of the National Science Foundation in the federal 
management of scientific and technical information (STINFO) was given, with an 
emphasis on understanding present activities through an historical perspective. 
Discussed were STINFO' s functions, authorities, accomplishments, and problems. 
Among the appendices were a list of acronymns for key STINFO organizations, a 
chronology of selected events, synopses of major reports, and science informa- 
tion program monetary obligations. 

U.S. Congress, Senate, Committee on Labor and Public Welfare. Federal Manage - 
ment of Scientific and Technical Information (STINFO) Activities; The Role 
of the National Science Foundation . Washington, D.C.: Government Printing 

Office, July 1975 (Avail. NSTIF N75-28954) . 

The study examined the role of the Federal Government, in particular, 
the National Science Foundation, in managing and monitoring scientific and tech- 
nical Information (STINFO) activities in both the public and private sectors. 

The study contained an overview of the conceptual and factual information essen- 
tial to an understanding of this complex subject area, including a review of 
salient developments during the past two decades. An analysis of the role per- 
formed by the National Science Foundation in encouraging and often funding 
selected STINFO systems and services was presented. The study provided a retro- 
spective look at the full range of studies, policy-level decisions, and organi- 
zational actions affecting the evolution of the STINFO community in the period 
1950-1975. Congressional efforts to stimulate, support, and occasionally scru- 
tinize these scientific and technical information services were presented. 

Whalen, Bruce K., and others. Scientific and Technical Information; Options 
for National Action . McLean, Va.: MITRE Corp. , November 1976 (Prepared 

for National Science Foundation; Avail. NTIS PB 281-863 and ERIC 
ED 1 35-385) . 

An analysis of Public Law 94-282, the National Science and Technology 
Policy, Organization, and Priorities Act of 1976, was conducted to identify 
major scientific and technical information issues and action alternatives for 
the Office of Science and Technology Policy. Past reviews and studies were 
used as the basis of the analysis. 
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winter, Donald C., and others. A Technology Assessment of Advances in Scien- 
tific and Technical Information Services . Pasadena, Calif.: Xerox 

Electro-Optical Systems, February 1977 (Prepared for National Science 
Foundation; Avail. NTIS PB 265-839) . 

This report postulated advances in scientific and technical information 
(STI) services in the following fields; microform publications, full-text 
files, machine-readable submission for publication, copyright changes, demand 
publishing, library automation, and high-speed high-quality facsimile. Possible 
secondary and higher order impacts of such advances were identified and weighed 
as to whether they were of concern or no concern to society. Impacts of concern 
to society were placed in three categories: those for whom action was required, 

was desirable, or was optional. Possible corrective action was identified in 
each case. The report also included sections showing the possible long-term 
Impacts of technological advances and making Impact chains more concrete in 
terms of recent public concerns. A summary of the current (1975) technology 
applicable to STI services was presented. 
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INFORMATION SYSTEMS - DESIGN AND EVALUATION 


Ackoffr R. L., and others# The SCATT Report; A Tentative Idealized Design of a 
National Scientific Cqnrounlcation and Technology Transfer System . 
Philadelphia, Pa.: University of Pennsylvania, July 1975 (Avail. 

NTIS PB 247-242.) 

The research team developed an ideal system for the scientific and tech- 
nical information enterprise in the United States. The methodology was based 
on successive revision of a conceptual framework for organizing the flow of 
information from points of origin to all possible points of application. One 
of the advantages of this approach was that all affected parties — information 
generators and users as well as information processors — can help shape the 
evolving model. 

Battelle's Columbus Laboratories. Development and Assessment of Scenarios for 
the Scientific and Technical Information Search System of the Future, 
Executive Summary . Columbus, Ohio: Battelle's Columbus Laboratories, 

April 1977 (Prepared for National Science Foundation; Avail. NTIS 
PB 268-711 ). 

Four scenarios for the STI search system of the future, each assuming dif- 
ferent combinations of levels of the environmental parameters — technology 
utilization, information priority, and c(xnpetitlon — were developed and assessed 
in this study. The resulting scenarios portrayed futures ranging frcan highly 
advanced technology-oriented systems to systems showing little technological 
progress with even deterioration of current levels of services. From the 
assessments it was indicated that information priority was the dominating 
environmental parameter. Progress depended on the support of the decision- 
makers and funders in government and industry and on the information demands 
of the scientific, engineering, and academic communities. Any necessary tech- 
nology was forecast to be readily available. 

Borman, Lorraine, and Dominick, W. Profile Evaluation, Research and Modelling 
for Science Information Systems: A Report on the Developnent of a 

Generalized Evaluation Methodology to Study User Interaction. Final 
Report . Evanston, 111.: Northwestern University, June 1978 (Avail. 

NTIS PB 283-979). 

The behavioral and performance patterns of individuals using interactive 
information systems were studied to identify factors which served to improve 
use of such systems and develop a generalized evaluation methodology. Data 
collection was accomplished through automated interaction monitors (operating 
in a relatively unobtrusive mode to the users) within two information systems 
at different universities, with many data bases and a broad user community. A 
series of questions covered such factors as system usage, errors and error 
recoveries, user experience, and success and satisfaction with the system. 
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Browiif Raymond N./ and Agrawalaf Ashok K. "On the Behavior of Users of the 
MEDLINE System." Changirtg Patterns in Information Retrieval . Tenth 
Annual National Information Retrieval Colloqulm. Edited by Carol 
Fenicke. Washington, D.C.: American Society for Information Science, 

1974, 36-38. 

Data were gathered in connection with an ongoing performance study of 
MEDLINE, an interactive version of MEDLARS (medical literature analysis and 
retrieval system) of the National Library of Medicine. The MEDLINE data base 
consisted of about half the citations which were published in the last three to 
four years of Index Medlcus. These citations contained bibliographic descrip- 
tions emd assigned index terms frcan articles in the biomedical fields. In 
three-quarters of the search sessions, the original requestor was not present. 
When present, he was usually assisted by a trained search analyst. Quantita- 
tive observations of terminal usage were discussed. 

Cleverdon, Cyril W. "Design and Evaluation of Information Systems." In (1971) 
Annual Review of Information Science and Technology . Vol. VI. Edited by 

Carlos A. Cuadra. Chicago: Encyclopedia Brittanica, Inc., 1971, 41-73. 

( 

This chapter presented a review of selected new literature in 1970 
(81 citations) on the design and evaluation of information systems. Papers were 
grouped and discussed in topics, including review papers, methodology, theo- 
retical investigations, experimental evaluation tests, development tests inves- 
tigating the optimum design in a given situation, tests of operating systems, 
cost effectiveness, and on-line systems. 

Committee on Information Hang-Ups. Users Look at the POD Information Transfer 
Process . Washington, D.C.: Ccmunittee on Information Hang-Ups, January 

1 975 (Avail. NTIS AD-A005400) . 

The Committee on Information Hang-Ups examined Defense Documentation Center 
(DDC) services to information users and looked at some aspects of the informa- 
tion transfer process within the Department of Defense (DoD) . While minor 
aspects of DDS's operations were critized, users felt that no major changes 
were required, although greater efforts needed to be made to educate users in 
what DDC offers. Serious obstacles to the information transfer process within 
DOD were identified. A substantial part of the difficulties found in transfer- 
ring information arose because there was no coordination of the process among 
divisions of the Executive branch of the government. 

Cooper, William S. "On Selecting a Measure of Retrieval Effectiveness." 

Journal of the American Society for Information Science , XXIV (March/April 
1973), 87-100. 

In Part I (JASIS, March-April 1973) the author proposed a methodology for 
evaluating a retrieval system based on how much users were willing to pay to 
use the system. Part II set forth steps in testing the methodology through the 
analysis of underlying assumptions and through experimentation. 
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Corriodore/ Michael C. Scientific and Technical Information Needs of Users or 
Potential Users of PSA-Administered POD Information Analysis Centers 8 
Final Report . Alexandria^ Va.: Defense Supply Agency/ 1976 (Avail. 

NTIS AD A024-937) . 

Two separate but related survey efforts were conducted. The first effort 
was aimed at determining the level of satisfaction of DoD Research and Develop- 
ment activities with the DSA-administrered DoD Information Analysis Centers 
(lACs). The second survey effort attempted to identify specific/ job-related 
scientific and technical information needs of Defense scientists and engineers 
which could be incorporated into the DSA-administered DoD lACS Five-Year plan 
as a set of long-range/ technical objectives. Principal findings from this 
effort included (1 ) three of every four Defense scientists and engineers 
experience problems in locating/ obtaining/ and using scientific and technical 
information/ (2) the most important products or services that could be produced 
or offered by the lACs were handbooks and databooks/ and (3) DoD scientists and 
engineers expressed a need for scientific and technical information in over 
thirty-five different areas. 

Debons/ Anthony/ and Montgomery/ K. Leon. "Design and Evaluation of Informa- 
tion Systems." In (1974) Annual Review of Information Science and 
Technology . Vol. IX. Edited by Carlos A. Cuadra. Washington/ D.C.: 

American Society for Information Science/ 1974/ 25-55. 

Developments in the design and evaluation of information systems over the 
period 1971-1973 were reviewed in this chapter. Major topics were theoretical 
developments/ status of operational advances in design and evaluation/ manage- 
ment systems/ communication systems, information networks, and important trends 
and conclusions. The findings of the 1 04 publications cited were interpreted 
in relation to their significance in information science. 

Dewhirst, H. Dudley. The Role of the LRC Technical Library in Fulfilling the 
Information Needs of Professional Employees . Washington, D.C.: National 

Aeronautics and Space Administration, August 1970 (Avail. ED 066-174). 

This library evaluation was based primarily on responses of sc«ne 320 mem- 
bers of the professional research staff and 50 members of the professional 
administrative staff at the Langley Research Center to a questionnaire. Evalua- 
tion first considered the library as only one part of a total system of informa- 
tion sources serving professionals. This was followed by an evaluation of the 
collection, facilities and tools, and services of the library based on responses 
of those who used the library more than six times a year. A major feature of 
the evaluation was the use of a critical incident technique in which respondents 
furnished details of a recent incident when they located information which 
proved, useful in their work. - 


34 



Franz, C. R. "Contingency Factors Affecting the User Involvement Role in the 
Design of Successful Information Systems." Ph.D. dissertation, Nebraska 
University (Avail. University Microfilms, 80-01065) 

A field survey was conducted to test various contingency hypotheses. Dif- 
ferent types of user involvement in designing and implementing information sys- 
tems were examined along with the resultant success of the information systems 
to the user. User involvement was studied under various organizational and 
user conditions to determine the effect which these various conditions had upon 
user involvement as well as the subsequent user -perceived usefulness of the 
information system. It was concluded that user involvement was an important 
variable in producing successful information systems. It was emphasized that 
more attention should be devoted to developing other measures of user involve- 
ment and system's success. 

Fry, Bernard M. Evaluation Study of ERIC Products and Services. Final Report . 
Blo<xnington, Ind.: Indiana University, Graduate Library School, 4 volumes 

and a summary volume, March 1972 (Avail. ERIC ED 060-923, 060-924, 060-925, 
060-926, 060-922) . 

This study examined the use of Educational Resources Information Center 
(ERIC) products and services by members of the educational community and 
evaluated the extent to which the ERIC system was achieving its objective of 
guaranteeing ready access to the nation's current significant literature in the 
field of education. The introduction, summary of findings, and recommendations 
were contained in both volume I 2 ind the summary volume. Also contained in 
volume I were discussions of the characteristics of ERIC users and demonstra- 
tion of the extent to which ERIC was meeting its goal of making significant but 
previously unavailable documents easily available to the educational community. 

Discussed in volume II were the ERIC Clearinghouses' information analysis 
activities, ERIC's efforts to help strengthen existing communication channels, 
ERIC activities designed to contribute to the development of a national infor- 
mation network, and profiles of organizations which provided ERIC services and 
products. Volume III described the methodology of this study and the question- 
naires used, summarized the conclusions and recommendations of ERIC study advi- 
sory panels, and provided background information. Volume IV was an appendix 
containing supplementary and parallel tables keyed to Chapters 4-7 of volumes I 
and II. The summary volume contained the introduction, summary of findings, and 
recommendations. The findings called attention to conditions, trends, and issues 
concerning use and user reactions to provide a concise, analytical basis for 
evaluating how well ERIC had met its goals and to identify deficiencies and 
weaknesses. The recommendations proposed improvements and courses of action to 
correct deficiencies identified by this study. 



Greenberg, Bette, and others. "MEDLINE Demand Profiles: An Analysis of 

Requests for Clinical and Research Information." Bulletin of the Medical 
Library Association , LXV (January 1977), 22-30. 

The results of a statistical analysis of professional use of the National 
Library of Medicine's bibliographic retrieval system, MEDLINE (Medical Litera- 
ture Analysis and Retrieval System On-Line) , at a large medical school library 
were presented. Results indicated that (1) demand for MEDLINE service was pri- 
marily research oriented, (2) frequency of use bore a relationship to rank and 
departmental affiliation, (3) broad and comprehensive searches were requested 
more frequently than searches for specific information, (4) usage showed an 
interesting curvilinear relationship with age and status of the user, and 
(5) grant funds and support correlated with the number of searches requested. 
Implications of these findings were that since clinicians' use of MEDLINE was 
found to be minimal, information services should be reevaluated in order to 
assist in meeting their information needs more effectively. 

Grashof, John P. An Operational Experiment for the Marketing of Scientific 
and Technical Information Innovations . Final Report . Harrisburg, Pa.: 
Pennsylvania Science and Engineering Foundation, May 1976. (Avail. NTIS 
PB 261-578.) 

The application of modern scientific marketing research technology to 
the scientific and technical information (STI) industry is described through 
one case study. The identification of those involved in the purchase and use 
of STI within an organization, their roles, and purchase patterns were investi- 
gated. Special attention was focused on users knowledge of STI systems and 
how that knowledge was applied to purchase decisions. A total of 274 inter- 
views with 1 71 firms were conducted to determine those within an organization 
who make STI purchase decisions. Small, medium-sized, and large firms were 
included in the sample. The results suggest that STI designers should con- 
sider offering a number of products to match the needs of various industrial 
segments. 

Henderson, Madeline M. Evaluation of Information Systems - A Selected 

Bibliography with Informative Abstracts . Washington, D.C.: National 

Bureau of Standards, (NBS TN-297) , December 1967 (Avail. NTIS PB 188-657; 
ERIC ED 061 -497) . 

A survey of the literature on evaluation of information systems was con- 
ducted. During the early stages of the survey, the literature was divided 
among descriptions of programs which compared the performance of two or more 
information systems, accounts of programs which studied the performance of one 
system, papers and reports which discussed the problems of evaluation programs, 
and documents which proposed new techniques for evaluation of systems. Prom 
the total literature collected, those references which were judged to be most 
useful were abstracted. The abstracts were designed to give a summary of the 
content of the corresponding paper; the author's own wording was used exten- 
sively, in order to avoid misinterpretations. All the references collected 
were listed, in alphabetical order of author's names, in the appendix to the 
main body of the publication. 



Hensley, Susan. Evaluation of Users' Resiaonses to the Information Analysis 

Products of the ERIC Clearinghouse on Tests, Measurements, and Eval uation. 
Princeton, N.J.: National Institute of Education, March 1978 (Avail. 

ERIC ED 1 59-21 7) . 

To obtain user evaluations of the information analysis products (lAPs) 
produced by the Educational Resources Information Center Clearinghouse on Tests, 
Measurement, and Evaluation (ERIC/TM) , questionnaires were administered to sub- 
jects randomly drawn frcro a 1976 list of the clearinghouse's lAP orders. Indi- 
cators of quality were document length, organization, writing, and format. 
Utility was judged according to the purpose for ordering the document, whether 
the document was passed on (if so, to whom and for what), and the document's 
impact on the requestor's subsequent work activities. User evaluations of 
ERIC/TM' s products tended to be high and indicated that the publications were 
used as reference tools by educators to keep abreast of the literature. The 
questionnaire and resulting data were appended. 

Herner, Saul, and others. An Evaluation of the Goddard Space Flight Center 
Library . Washington, D.C.: Herner and Co., July 1979 (Avail. NTIS 

N80-251 98; NASA CR-1 59-969). 

The character and degree of coincidence between the current and future 
missions, programs, and projects of the Goddard Space Flight Center and the 
current and future collection, services, and facilities of its library were 
determined from structured interviews and discussions with various classes of 
facility personnel. In addition to the tabulation and interpretation of the 
data from the structured interview survey, five types of statistical analyses 
were performed to corroborate (or contradict) the survey results. 

Katter, Robert V. "Design and Evaluation of Information Systems." In (1969) 
Annual Review of Information Science and Technology . Vol. IV. Edited by 
Carlos A. Cuadra. Chicago; Encyclopedia Britannica, Inc., 1969, 32-70. 

Most system development activities were not associated with a brand new 
system totally divorced from any previous system, but rather with the replace- 
ment of an older system one that was newer, usually more expensive, and pre- 

sumably more effective. The design and evaluation literatrure of 1 968 was 
reviewed in the context of such a system redevelopment cycle. One hundred 
eighty-one references were discussed under the following topics: the system 

redevelopment process (planning and modeling aids, textbooks, and management); 
the system's demand environment; the systems's support resources; the system's 
operational constraints (copyright, data file security, standardization, func- 
tional overlap) ; cost-effectiveness analysis (measures of retrieval effective- 
ness, error sources in search accuracy measurement, search accuracy studies, 
man-system interface arr 5 uigements, hardware, and software) ; and conclusions. 
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King, Donald W. "Design and Evaluation of Information Systems." In (1968) 

Annual Review of Information Science and Technology , Vol. III. Edited by 
Carlos A. Cuadra. Chicago: Encyclopedia Britannica, 1968, 61-103. 

The 1 967 literature indocumentation and information retrieval was reviewed 
and discussed (201 references) . Topics covered were: information systems con- 

cepts and analytical techniques, measures and criteria for design and evaluation 
defined by costs, measures of system benefits, measures of system performance, 
test and evaluation methodology, and results of test and evaluation in search 
and retrieval subsystems and announcement subsystems. 

King, Donald W., and Bryant, Edward C. The Evaluation of Information Services 
and Products . Washington, D.C.: Information Resources Press, 1971. 

This textbook addressed the evaluative and quality control aspects of 
library and information system design and operation. It covered the evaluative 
and control aspects of: classification and indexing processes and languages; 
document screening processes; composition, reproduction, acquisition, storage, 
and presentation; and user-system interfaces. The book also contained brief 
primers on user surveys, statistics, sampling methods, and experimental design. 

King, Donald W., ed. Key Papers in the Design and Evaluation of Information 

Systems . White Plains, N.Y. : Knowledge Industry Publications, Inc., 1978. 

Evaluation has played a particularly important role in the evolution of 
information systems over the past two decades. During the early and mid-1960's, 
a large amount of funds were expended in the United States on innovations in 
information retrieval systems. In the 1970's, the focus changed to operations 
of library systems, networks of libraries, and networks of information retrieval 
systems. The evaluation that had taken place over the years had influenced spe- 
cific system designs and the results of past evaluation studies had affected the 
way in which system designs were implemented. This volume provided a single 
source of recent and/or classic papers for students, instructors, and practi- 
tioners who wished to use or study this facet of information science. 

Krevitt, Beth I., and Griffith, Belver C. "Evaluation of Information Systems. 

A Bibliography, 1967-1972," Information Part 2; Reports Bibliographies , 

II (6) 1973 . 1-34 (Avail. ERIC ED 083-988). 

The scope of this bibliography was limited to literature dealing with the 
design, testing, and evaluation of information storage and retrieval systems. 

A number of papers describing the implementation of specific systems were 
selectively included because they contained substantial or innovative material 
on evaluation; however, the bibliography generally emphasized techniques applic- 
able to a- wide variety of systems. Similarly, articles dealing with specific 
products and services were included when they showed broad implications for 
evaluation. On the other hand, discussion of specific library systems were 
excluded as they were considered outside the scope of the bibliography. 



Lancaster, P. Wilfrid. Information Retrieval Systems.; Characteristics, 

Testing and Evaluation . New York: John Wiley and Sons, 1968. 

Indexing policy and practice, vocabulary control, searching strategies, 
and user-system interactions were represented as factors which significantly 
affected the performance of all information retrieval systems. Only subject 
retrieval systems were actually discussed. Topics covered in 1 6 chapters 
included: subject indexing, using both traditional and concept coordination 

retrieval tools; search files and searching methods; current awareness; per- 
formance of retrieval systems; index languages (complete sets of terms used 
to describe document content) ; approaches to mechanized systems (statistical 
analysis, automatic assignment of terms and classifications, automatic search- 
ing); evaluation of operating efficiency and economic efficiency; and user 
interaction with systems. 

Lancaster, P. Wilfrid, and Gillespie, Constantine J. "Design and Evaluation of 
Information Systems." Annual Review of Information Science and Technology . 
Vol. V. Edited by Carlos A. Cuadra. Chicago: Encyclopedia Britannica, 

Inc., 1970, 33-70. 

This chapter reviewed selected 1 969 publications on design and evaluation 
of information systems, particularly those papers illustrating important trends. 
Topics covered were: national and international information systems and net- 

works; system design (components, fully automatic systems, and on-line inter- 
active systems); personal information files; relevance and relevance predicta- 
bility; and evaluation of complete systems, system components, published 
indexes, literature usage, library services, human factors, and economic 
efficiency. (134 references) 

Linguist, Mats G. "Dynamic Modeling of Information Services: Project Overview." 

Information Revolution. Proceedings of the 38th Annual Meeting of the 
American Society for Information Science. Edited by Charles W. Husbands 
and Ruth L. Tighe. Washington, D.C.: American Society for Information 

Science, 1975, 43-44. 

This study was aimed at a comprehensive analysis of the interaction between 
the users and the information service. The methodology chosen was that of sys- 
tem dynamics. The thrust was to analyze the structure of systems, and to iden- 
tify possibilities for improving system performance. The developed model (given) 
was oriented towards the management of an information service. 



Liston, David M. , Jr., and Schoener, Mary L. "A System Approach to the Design 
of Information Systems." Journal of the American Society for Information 
Science , XXII (2) (March/April 1971 ) 11 5-122. 

A systems approach to the design of document-handling information systems 
required a detailed examination of the choices made in the design process and 
the ramifications of possible choices in terms of the capabilities, performance, 
cost, and other characteristics of the system. The authors advocated a sys- 
tematic procedure involving six steps (1 ) indent ification of fixed parameters, 
(2) identification of variable parameters, (3) identification of available 
options for each variable parameter, (4) identification of factors affecting 
a choice among available options, (5) identification of factors affected by a 
choice among available options, and (6) logical analysis of the picture thus 
presented. The analysis determined the optimum sequence for decision during 
the design process and the nature of the decision process itself. 

Lucas, Henry C., Jr. "Systems Quality, User Reaction, and the Use of 
Information Systems." Management Information , III, 1974, 207-212 
(Avail. ERIC ED 089-771 ), 

One hundred seventeen administrators, users of both batch process and on- 
line computer systems at a major university, responded to a questionnaire 
designed to discover attitudes toward the quality of the system and services, 
the users' evaluation of the service, and the relationship of these attitudes 
with voluntary use of the system. These attitude, evaluation, and use factors 
were considered in relation to the actual quality of the system as rated inde- 
pendently by the information services department staff. Data were collected 
on input-output quality, adequacy of training, management support, quality and 
kinds of reports generated, input error ratings, usefulness, accuracy, timeli- 
ness, flexibility, completeness, and pressure to reprogram the system. The data 
showed an association between actual quality of systems and quality as perceived 
by users with favorable user attitude. These favorable attitudes and evalua- 
tions were also positively associated with system usage. Further increased 
voluntary use may be predicted by favorable attitudes, which provided good 
reasons for information services to develop high quality systems and favorable 
user attitudes. 

McCarthy, Susan E., and others. "Evaluation of MEDLINE Service by User Survey." 
Bulletin of Medical Library Association , LXII (October 1974), 367-373. 

An evaluation of the MEDLINE service at the Calder Memorial Library of the 
University of Miami School of Medicine was conducted. A one-page questionnaire 
was sent to 350 patrons who had requested MEDLINE searches in 1973. The 
response validated many assertions about the user group and their reasons for 
using MEDLINE. Quite surprising were the degree of enthusiasm, the willingness 
to pay out of personal funds, the apparent lack of knowledge about SDILINE, and 
the number of critical comments. The experience gained by this assessment has 
suggested improvements in some aspects of service, as well as more subject- 
specific evaluations at intervals on the future. 
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Milcarek, Barry I., and Struening, Elmer L. "Evaluation Methodology: A 

Selective Bibliography." Handbook of Evaluation Research . Vol. I. 

Beverly Hills, Calif.: Sage Publications, Inc., 1975, 667-676. 

In choosing the bibliography entries, the focus was on general issues in 
the methodology of program evaluation. Writings concerned with describing 
methodological resources, their limitations, and the modifications demanded by 
practical applications were given most serious attention. References were 
arranged under the following headings: issues in conceptualization, issues in 

measurement, issues in design, and issues of interpretation. 

Moll, Wilhelm. "MEDLINE Evaluation Study." Bulletin of the Medical Library 
Association , LXII (January 1974), 1-5. 

MEDLINE (MEDLARS-on-line) is an on-line, interactive bibliographic search- 
ing system developed by the National Library of Medicine. The system provides 
users with lists of bibliographical citations and other information from a file 
of over 1,250 bio-medical journals. A survey testing user reactions was con- 
ducted at the University of Virginia Medical Library. The results of the sur- 
vey were based on replies by 246 users who requested one or more MEDLINE 
searches between September 1972 and March 1973. The findings indicated that 
more than 93% believed the MEDLINE was a substantial improvement over the 
traditional methods of searching through the printed indexes. These respondents 
also stated that the results of MEDLINE searches had assisted them in their 
clinical or research work, or both. Asked whether they would continue to use 
MEDLINE after the imposition of user charges on July 1, 1973, about 75% said 
that they would. Since the remaining 25% expressed some reservations and doubts 
a decline in the use of MEDLINE was projected once user charges were imposed. 

Pinelli, Thomas E., and others: A Review and Evaluation of the Langley 

Research Center's Scienctific and Technical Information Program : Results 

of Phase I - Knowledge and Attitudes Survey, LaRC Research Personnel . 

NASA TM-81893. Washington, D.C.: National Aeronautics and Space 

Administration, November 1980, (Avail. NTIS Pending). 

The results of Phase I - Knowledge and Attitudes Survey, LaRC Research 
Personnel were contained in this report. Phase I was conducted as part of the 
Langley STI Review and Evaluation Project. The purpose of Phase I was to 
assess the effectiveness of the Langley STI program based on feedback obtained 
frcan Langley engineers and scientists. Phase I utilized survey research. The 
survey procedure was conducted in two stages. Stage one involved personal inter 
views with 64 randomly selected Langley engineers and scientists. The inter- 
views were used to obtain information for questionnaire development. Stage two 
involved the collection of data through the construction and administration 
of a questionnaire. The questionnaire, which was based on information gathered 
in stage 1, covered various aspects of the Langley STI program, utilized both 
open- and closed-ended questions and was pretested for finalization. The 
questions were organized around the six objectives for Phase I. From the 647 
ccwipleted questionnaires, a randcsn sample of 300 were analyzed. Prom the 
analysis of the data, recommendations were made for improving the Langley STI 
program. 
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Pinelli, Thomas E., and others. A Review and Evaluation of the Langley 
Research Center's Scientific and Technical Information Program ; 

Results of Phase IV - Knowledge and Attitudes Surveyy Academic and 
Industrial Personnel . NASA TM-81 934. Washington, D.C.: National 

Aeronautics and Space Administration, February 1981, (Avail. NTIS 
Pending) . 

The results of Phase IV - Knowledge and Attitudes Survey, Academic and 
Industrial Personnel were contained in this report. Phase IV was conducted as 
part of the Langley STI Review and Evaluation Project. The purpose of Phase IV 
was to assess the effectiveness of the NASA and Langley STI program based on 
feedback obtained frcan engineers and scientists frcam the academic and industrial 
community. Phase IV utilized survey research. The survey procedure was con- 
ducted in two stages. Stage one involved personal telephone interviews with 
randomly selected scientists and engineers within the United States (Alaska and 
Hawaii excluded) . Stage two involved the collection of data through the admin- 
istration of a questionnaire. The questionnaire covered various aspects of the 
NASA and Langley Research Center STI programs and utilized both open- and closed- 
ended questions. The questions were organized around seven objectives for 
Phase IV. Frcxn the 492 mailed questionnaires, 381 valid returns were analyzed. 
From the analysis of the data, recommendations were made for improving the 
NASA and Langley STI program. 

Pryor, Harold E. "An Evaluation of the NASA Scientific and Technical Infor- 
mation System." Special Libraries , LXVI (11), (November 1975), 51 5-51 9. 

NASA interviewed users of its scientific and technical information system 
in order to assess the usefulness of the present system and to find ways to make 
it more effective. Emphasis was on announcement and current awareness services, 
the publication program, and the interactive retrieval system. 

Pryor, Harold E. "Listening to the User," In AGARD The Problem of Optimization 
of User Benefits in Scientific and Technological Information Transfer . 

Paris, France: AG»WD, March 1976, (Avail. NTIS N76-25109). 

A system evaluation study of the NASA scientific and technical information 
system was described. The thrust of this examination was to evaluate the use- 
fulness of the present information system to those it was intended to serve 
(engineers and scientists working in their professional roles) and to identify 
areas and ways in which the system could be made more responsive to user needs. 
Techniques used covered the range fran personal, in-depth interviews to widely 
distributed questionnaires. The findings were positive. Many refinements made 
to on-going programs and projects and new endeavors begun in direct response to 
stated user needs were discussed, in the main, these needs were not unreason- 
able and thus could be responded to with cost effective system modifications. 
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Rees, Alan M. "Evaluation of Information Systems and Services." In (1967) 

Annual Review of Information Science and Technology . Vol. II. Edited by 
Carlos A. Cuadra, New York, John Wiley and Sons, 1967, 63-86. 

This chapter summarized and discussed the state of the art in testing and 
evaluation in 1967. The few substantive research projects involving testing 
and evaluation were outlined in some detail. Although a distinction was made 
in this review between laboratory-based experimentation and tests of operational 
systems, the methodology used in each instance was substantially the same. A 
number of research projects in areas cognate to testing and evaluation were 
described, and s(xne general conclusions were provided with respect to past and 
future activity. (52 references) 

Scott, Helen B. "The Development and Administration of a System: The Dissemi- 

nation of Scientific and Technical Information." Unpublished Ph.D. 
dissertation, Texas A & M University, 1978 (Avail. University Microfilms, 
79-09235) . 

A Study to develop a more efficient system for the dissemination of 
scientific and technical information which could be adapted to other users with 
similar information needs on a national and international basis was conducted 
in the Department of Agricultural Communications, Texas A & M University. 
Publications of the Texas Agricultural Experiment Station (TABS) were used. 

Two phases were developed for the dissemination system: (1 ) an abstract noti- 
fication system for notifying the TABS distribution list that a publication was 
available and (2) a brief publications format more comprehensive than an abstract 
but less comprehensive than the full text of the publication. 

Severy, Lawrence J. Evaluation: Problems and Prospects . Princeton, N.J.: 

National Institute of Education, November 1975 (Avail. ERIC ED 117-190). 

This publication presented a comprehensive but non- technical discussion of 
the issues surrounding program evaluation and the experimental method. Methods 
for effectively employing program evaluation were suggested to administrators, 
educators, graduate students, novice researchers, and program and project 
directors. Discussions included (1) the reasons for applying the experimental 
method to program evaluation, (2) the basic elements of the experimental method, 
(3) refinements and variants of the experimental method with examples from 
program evaluation in the social sciences, (4) ways in which funding directives 
can be adapted to the experimental method, and (5) potential problems connected 
with the experimental approach. 
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Smith# William A.# Jr. Studies in Evaluation of Information System Effective- 
ness . Bethlehem# Pa.j Leigh University, Department of Industrial Engi- 
neering, June 1972 (Avail. NTIS AD 744-027). 

The varied characteristics and purposes ascribed to information systems 
were identified# including the requirement to support the decision process at 
one or more levels of activity. The value of information and its impact on 
organizational objectives were discussed. A proposed information-decision 
model stressed the recurring# dynamic demands for system output. Measures of 
effectiveness were derived from functional objectives in both qualitative and 
quantitative terms. Economic and behavorial influences on these performance 
indicators were considered. 

Smith# William A.# Jr.# and Wechsler# Ben L. Planning Guide for Information 
System Evaluation Studies . Norcross# Ga.: American Institute of 

Industrial Engineering# Computer and Information Systems Division# 

August 1973. 

This monograph proposed a comprehensive# systematic approach to the plan- 
ning of evaluation studies for information systems. It suggested a careful 
analysis of the reasons for such a study; the nature of the system#' and the 
functional# user-oriented requirements which were to be supported. The Issues 
related to assessment of information processing activities and to definition 
of measures of effectiveness were discussed. Emphasis was placed upon identi- 
fying tangible benefits wherever possible. Organization of evaluation studies 
was covered and an illustrative example was provided. 

Struening# Elmer L.# and Guttentag# Marcia# eds. Handbook of Evaluation 
Research. Beverly Hills# Calif.; Sage Publications# Inc.# 1975. 

One purpose of this handbook was to provide a reference which evaluators 
could consult as they developed their studies. The second purpose was to serve 
as a textbook for courses in evaluation at the graduate level. Volume I# 
divided into eight parts# focused on some aspect of the evaluation process. 

Papers were compiled under the following headings; preface# policy and strategy 
in evaluation research# conceptualization in design of evaluation studies# devel- 
opment and evaluation of measures# data collection through social ecology# data 
analytic methods# and communication of evaluation results. Volume II focused on 
evaluations in context. Several content areas# their special problems# and the 
evaluation methodologies applicable to them were presented and discussed. Papers 
were presented under the headings; preface, politics and values in evaluation 
research# cost-benefit approach to evaluation# evaluation of mental health pro- 
grams# and selected content areas in evaluation research. 
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Swansorif Rowena W. "Design and Evaluation of Information Systems." In (1975) 
Annual Review of Information Science and Technology . Vol. X. Edited by 
Carlos A. Cuadrar Washlngtonr D.C.: American Society for Information 

Sclencer 1975^ 43-101. 

The literature selected for this chapter consisted of 1 973 and 1 974 pub- 
licationSf which were reviewed and discussed under the following sections: 
design of information systemsr methodological approaches to evaluation^ evalua- 
tions of information systems f performance measures/ and evaluation method- 
ologies of information science. The design section was subdivided to reflect 
both outcomes of design activities and appraisal components of the design 
process. The second section provided descriptions of seven major evaluation 
methodologies that have been explored and developed by non-information-science 
groups. Evaluations within information science were treated primarily frcxn a 
methodological viewpoint. (276 references) 

Swanson/ Rowena Weiss. "Performing Evaluation Studies in Information Science." 
Journal of the American Society for Information Science . XXVI (3)/ (May- 
June 1975)/ 140-156. 

This paper considered conceptual and methodological c(xnponents of informa- 
tion science evaluation studies. The paper discussed the judgmental process 
of evaluation and the scientific nature of evaluation study in the context of 
purpose statements; criteria; the selection of variables and data collection 
and analysis techniques; and requirements of validity/ reproducibility and 
reliability. Industrial value analysis/engineering methodology was described 
and related to assessments of information products and services. The state-of- 
the-art of evaluation study in information science was analyzed with respect 
to (1) the scope of evaluation studies/ (2) the use of laboratory type environ- 
ments/ (3) the use of surrogate judges/ (4) selection of variables/ (5) fre- 
quency of study/ and (6) comparability of study results. Evaluation study was 
seen as essential to the management of information centers and systems and as 
having appreciable growth potential. 

Sweezy/ Eldon/ Hopper/ Janice. Obstacles to Innovation in the Scientific and 
Technical Information Services Industry . Washington/ D.C.: Institute 

of Public Administration/ October 1975 (Prepared for National Science 
Foundation; Avail. NTIS PB 253-449). 

The objective of this report was to evaluate the efficiency and effective- 
ness of the scientific and technical information services serving the needs of 
the state and local public sector decisionmakers/ with a view to enhancing 
service capabilities and improving state and local officials' understanding of 
what is available and how it serves them. The topics addressed were (1 ) how 
scientific and technical information requirements in the public sector differ 
from the needs of commercial and corporate userS/ (2) whether cost considera- 
tions affect public sector information users differently from private sector 
users/ and (3) how information users and suppliers evaluate all of the separate 
factors in formulating their plans. 
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Taube, Mortimer. An Evaluation of "Use Studies" of Scientific Information . 
Washington/ D.C.: Documentation/ Inc./ 1959. 

An evaluation of the total literature on use studies was attempted. The 
evaluation accepted the conclusions drawn by other surveys of use studies which 
appeared before the International Conference on Scientific Information in 
November 1 958 and brought those conclusions up to date by abstracting and 
evaluating the studies prepared for the conference. An attempt was made to 
analyze the reasons for the generally accepted failure of use studies by estab- 
lishing a distinction between consumer services and professional services. It 
was concluded that the organization and dissemination of scientific Information 
was a professional activity/ the value of which could not be measured by con- 
sumer responses/ and that such responses could not supply direction for the 
design of more effective scientific information and reference systems. 

Weiss/ Carol H. Evaluation Research: Methods for Assessing Progr^un Effective- 
ness . Englewood Cliff S/ N.J.: Prentice-Hall/ Inc./ 1972, 

This book dealt with the application of research methods to the evaluation 
of social programs/ such as programs in education/ social work/ health/ mental 
health/ and technical assistance. The basic theme of the book was that evalua- 
tion used the methods and tools of social research but applied them in an action 
context that was intrinsically inhospitable to them. A principal purpose was 
to alert readers to prevalent issues and problems and offer guidance for adapting 
textbook methods to real world evaluation settings. 

Wolek/ Francis W. "Professional Work: The Context for Evaluating the Impact of 

User Studies." Changing Patterns in Information Retrieval. Tenth Annual 
National Information Retrieval Colloquium . Edited by Carol Penickel. 
Washington/ D.C.: American Society for Information Science/ 1974/ 34-35. 

It was proposed that amount of usage was not the proper criterion for 
evaluating formal systems because only a small number of people in a field are 
actively involved in original work. Progress in many fields consisted of 
relatively small incremental advances within established lines of work. While 
interaction with formal systems constituted only a small part of users' total 
activities in acquiring information/ a strong need to understand the users' 
problems was predicted. 
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INFORMATION - USE AND NEED 


Advisory Group for Aerospace Research and Development. Scientific and Technical 
Information -- Why? Which? Where? and How? Paris, Prance: AGARD Lecture 

Series No. 44, February 1971 (Avail. NTIS N71 ”23501). 

This publication consisted of eight papers presented in an AGARD lecture 
series. The primary purpose of this series was to bring to the attention of 
users (technologists and scientists) , the present capabilities of information 
systems, services, and media retrieval techniques. The series covered the basic 
background on what information can do for scientists and technologists, the 
needs of the technical community, sources of technical information, information 
channels and formal services for obtaining information, and selective dissemina~ 
tion techniques. The concept, mission, and operation of scientific and techni- 
cal information analysis centers were also presented, and their relationship 
to conventional information centers was described. 

Advisory Group for Aerospace Research and Development. The Problem of Optimiza - 
tion of User Benefit in Scientific and Technological Information Transfer . 
Neuilly Sur Seine, Prance: Advisory Group for Aerospace Research and Devel- 

opment, 1978, AGARD Conference Proceedings No. 179 (Avail. NTIS N76-25098) . 

A summary of a panel discussion on the fifteen papers presented during the 
Technical Information Panel Specialists' Meeting was given. The papers were 
cncerned with investigating the user/service interface and defining users' prob- 
lems particularly for aerospace, scientific, and technical information. User 
population and their needs were examined with the view toward helping system 
designers and managers provide services to meet these requirements. 

Allen, Thomas J. "Information Needs and Uses." In (1969) Annual Review of 

Information Science and Technology . Vol. IV. Edited by Carlos A. Cuadra. 
Chicago: Encyclopedia Brittanica, Inc., 1969, 3-29. 

The literature appearing during 1 968 on information needs and uses was 
reviewed in the conceptual framework of the scientist as an information proces- 
sor and communicator within a work team, professional society, invisible college, 
and other organizations. Both formal structure and informal organization were 
important factors in communication patterns and information flow. 
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Allen, Thomas J. Meeting the Technical Information Needs of Research and 

Develop ment Projects . Cambridge, Mass.: Massachusetts Institute of 

Technology, November 1969 (Prepared for National Science Foundation; 

Avail. ERIC ED 047-748). 

Data from a large number of recent research studies were brought to bear on 
the problem of providing for the technical information needs of research and 
development projects. The importance of proper support by the technical staff 
was shown, and it was argued that the best way to couple the project team to 
information sources outside the organization was by an indirect route through 
key personnel among the laboratory's technical staff. Informal relations and 
physical location were shown to be important determinants of the structure of 
organizational communication networks. The effect of physical location on com- 
munications was especially strong and required serious consideration when 
designing research facilities. The formation of project teams and intergroup 
transfers and loans were presented as opportunities for the development of 
informal relations. 

Allen, Thomas J. "Organizational Aspects of Information Plow in Technology." 

Aslib Proceedings , XX (November 1968), 433-54. 

A study was conducted by a research group working at the Massachusetts 
Institute of Technology to explore the information needs of scientists and engi- 
neers, the means for meeting these needs, and interrelationships between differ- 
ent means and the efficiency of research work. Among the information channels 
investigated were literary sources, contractors, buyers, in-house studies, and 
laboratory staff. The utilization of these channels by specialists and the 
efficiency of information received through them were also studied. A major con- 
clusion was that internal information channels were more efficient than the 
external ones and that the majority of engineers made much less use of literary 
sources than might have been expected. On the other hand, every laboratory had 
on its staff a number of specialists who were particularly active users of 
external channels, kept abreast of periodicals in the field, and regularly read 
special literature. They served as information sources for their colleagues, 
who, as ascertained, often turned to them for consultation on various technical 
problems. The important role such specialists had to play in research was 
emphasized. 

Bates, Marcia J. User Studies: A Review for Libraries and Information 

Scientists. Bethesda, Md.: Leasco Information Products, Inc., 1971. 

This selective review covered user studies done from the standpoints of 
both the library and the user. Catalogs, reference services, circulation, 
browsing, and library facilities were included, as were information-gathering 
habits of scientists and the general public. Studies of whole information sys- 
tems and the methodology and philosophy of user studies were discussed. The 
coverap of 1 81 listed papers was restricted to those in the English language 
through 1968, with a number of 1969 papers included. 
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Barnes/ R. C. M. "Infocroatlon Use Studies: Ccinparlson of Scxne Recent Surveys." 

Journal of Documentation , XXI (1 965) r 169-176. 

Two information-use surveys (one conducted by the National Lending Library 
for Science and Technologyr and one conducted on behalf of the Advisory Com- 
mittee on Scientific Policy) were compared with each other and with earlier sur- 
veys in regard to sampling, design and format of the survey, analysis and 
reporting of data, and selected results. This examination showed that, while 
the results were not contradictory, differences in principle and method made it 
impossible to demonstrate close agreement. It was concluded that direct compar- 
ison of results from different surveys was difficult, and the superficial com- 
parisons could be misleading. 

Berrill, Allison, and Corbett, Lindsay. "User Needs and Information Supply: 
Report on the Scottish Branch Conference." Information Scientist , VII 
(March 1973), 20-22. 

A one-day conference on user needs and information supply was held at the 
University of Stirling (Scotland) on November 10, 1972. Brief summaries were 
given on the following papers: "Secondary Information Services and the User" 

by T. M. Aitchison; "User Needs in the Product Data Field" by W. Tyson; "Assess- 
ing User Needs in an Information Context" by J. L. Schofield; and "Some Economic 
Aspects of Information Supply" by J. N. Wolfe. 

Boies, Stephen J. User Behavior on an Interactive Component System . Yorktown 

Heights, N.Y.: Watson Research Center, August 1972 (Avail. NTIS AD-754836). 

The results of an analysis of user behavior on an interactive system were 
presented. Empirical data on user behavior were presented concerning: (1) the 

duration and frequency of user terminal sessions, (2) the use of language pro- 
cessors, (3) user response time, and (4) command usage. The results were dis- 
cussed in terms of the behavioral literature relevant to the design of interac- 
tive systems. Suggestions were made with respect to those areas which sould be 
investigated by behavioral scientists. 

Brittain, John M. Information and Its Users; A Review With Special Reference 
to the Social Sciences . Calverton Down, Bath, England; Bath University 
Press, 1 970. 

An extensive investigation of the literature and studies related to infor- 
mation requirements in the social sciences was conducted. The author concluded 
that user studies in the social sciences had; (1) only recently begun to appear, 

(2) drawn heavily on user studies in the sciences on methodological issues, and 

(3) been conducted without strong theoretical or conceptual frameworks. 
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Cleverdon, Cyril W. "User Evaluation of Information Retrieval Systems." 

Journal of Documentation / XXX (June 1974), 170-80. 

Recall and precision were established as the measures for determining the 
performance of a system. To determine the value or usefulness of a system to 
the actual users or to discover how to attract new users, the form of the evalu- 
ation must be user-oriented rather than management-oriented. Providing the 
basic operational characteristics of the system were known, three dimensions 
for evaluating the performance of a system in regard to user satisfaction were 
established: the quantitative measure of "search length," the qualitative mea- 

sure of "satisfaction," and the number of relevant citations consulted. Use 
of these dimensions would provide basic data for an analysis permitting either 
an increase in user satisfaction or a decrease in operating costs. 

Crane, Diana. "Information Needs of Users." In (1971) Annual Review of Infor - 
mation Science and Technology . Vol. VI. Edited by Carlos A. Cuadra. 
Chicago: Encyclopedia Brittanica, Inc., 1971, 3-39. 

This chapter reviewed 109 publications, mostly issued in 1969 and 1970, 
concerned with information needs of users. The first section examined literature 
on user needs in the basic sciences. Subtopics included the social organization 
of the research area, intellectual development of research areas, connections 
between research areas, diffusion of information from the research area to the 
formal communication system, information seeking, the scientists and the formal 
communication system, and innovations in formal communications systems; current 
controversies. Section two dealt with publications on information needs and 
uses in technology, and section three was concerned with literature on inter- 
national aspects of information needs and uses. 

Crawford, Susan. "Information Needs and Uses." In (1978) Annual Review of 
Information Science amd Technology . Vol. XIII. Edited by Martha E. 
Williams. White Plains, N.Y.; Knowledge Industry Publications, Inc., 

1978, 61-81 . 

This paper reviewed 95 publications on information needs and use studies, 
issued during the period 1975 through 1977, which had been identified from 
"Information Science Abstracts," "Library Literature," and informal sources. 

It addressed developments in four major areas: the concepts of information 

needs and uses, methodology of use studies, uses of information, and review of 
use studies published during 1975-1977. The use studies were categorized and 
discussed by area of focus: physical science and technology, biological/health 

sciences, social sciences, human information needs, mass media, use of specific 
techniques, and conceptual or review papers. 



Disch, A. "The Voice of the Users His Information Needs and Requirements 

(Which Are Not What the Information Specialists Thinks They Are)." The 
Problem of Optimization of User Benefits in Scientific and Technological 
Information Transfer . PariSr Frances AGARDf March 1976 (Avail. NTIS 
N76-25098) . 

Two user studies were carried out in Norway. In the first oner lr400 engi- 
neers were asked to list those information sources that they had used lately and 
which had proved to be of practical value in their daily work. The follow-up 
was a detailed study of the information gathering habits of a limited number of 
users. It was found in both studies that the traditional information channels 
were of little practical value to the user. Two attempts to remedy these find- 
ings were described. 

Faiblsoffr Sylvia G.r and Elyr Donald P. "Information and Information Needs." 
Information Reports and Bibliographies . V. N.Y.s Science Associates/ 
Inter nationalr Inc.f 1976, 2-16. 

After a conceptual framework in which information, information needs, and 
communication were defined and discussed, this paper reviewed the literature on 
studies of information needs in the areas of the sciences, the social sciences, 
the professions, and the general public. The emphasis of past studies was 
found to cluster in four groups (1) the behavior of the user; (2) the nature, 
amount and source of information being sought; (3) the quality of the informa- 
tion; and (4) the timeliness of the information. Given the findings of past 
needs studies, 12 guidelines for the design of user-oriented information sys- 
tems were presented. 

Forecasting International, Ltd. An Application of Market Research Techniques 
to the Dissemination of Scientific and Technical Information . Arlington, 
Va.! Forecasting International, Ltd., February 1977 (Avail. NTIS 
PB 265-130) 

The goal of the project was to develop a methodology for providing data to 
enable the producers and distributors of STI products and services to target 
their product design, distribution, and promotion efforts more effectively. 

The project was developmental, with a focus on knowledge about user character- 
istics in general. The methodology was based on the market research technique 
called market segmentation. Questionnaires were canpleted through personal 
interviews with a scientifically designed, nationwide sample of 402 chemists 
and electrical engineers. The respondents ranked nine types of information 
needs. The data analysis provided a description of six market segments in each 
discipline which had characteristics in common. The study validated the appli- 
cation of market segmentation to STI markets for improving user access to 
information. 
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Freeman/ James E., and Rubenstein, Albert H. The Users and Oses of Scientific 
and Technical Information; Critical Research Needs . Denver/ Colo.: Uni- 

versity of Denver Research Institute/ November 1974 (Prepared for National 
Science Foundation; Avail. NTIS PB 237-941 ) . 

To help identify what user and use problems should be given highest 
research priority in the near future/ the NSF arranged for a workshop in Denver 
in November 1973 at which 21 industrial/ governmental/ and academic representa- 
tives generated and set priorities on 50 critical research issues. Participants 
also completed two post-workshop evaluations of the original set of scientific 
and technical information user/use issues. Results of the workshop and the 
followup activity were presented in this report. 

Garvey/ William D./ and others. "The Dynamic Scientific-Information User." 
Information Storage and Retrieval / X (March/April 1974)/ 115-131. 

The concept of the Information user was examined, and data were discussed 
relative to "intraindividual variations" (changes which occur within individual 
scientists as their scientific work progresses) and "inter individual variations" 
(physical vs social scientists, basic vs applied scientists, experienced vs 
inexperienced scientists, and scientists working in the same subject-matter area 
vs scientists who recently changed their areas). All of these were found to 
produce significant variation in the information needs of scientists and in the 
sources they used to satisfy these needs. Some implications of these findings 
relative to information technology were discussed. 

Gellman, Aaron. The Role and Application of Scientific and Technical Informa - 
tion in the Process of Innovation: Invention and Conception. Final Report . 

Jenkintown, Pa.: Gellman Research Associates, Inc., December 1975 (Pre- 

pared for National Science Foundation; Avail. NTIS PB 256-580). 

The information gathering and use habits of a sample of scientists and engi- 
neers who, by job function and performance, were found to be innovators were 
explored. The data were collected for the study using a structured interview 
format with a number of open-ended questions. The survey was administered in 
person by a member of the project staff. Formal channels of communication, par- 
ticularly the STI subset, were not extensively employed by the innovators. 

The George Washington University Medical Center. Preliminary Report of the 

User's Perspective Conference of the National Forum on Scientific and Tech - 
nical Conmunication . Washington, D.C.: National Science Foundation, 

November 1 976 (Avail. NTIS PB 269-057) . 

Availability and utilization of scientific and technical information was 
assessed, by professionals from public and private organizations, public 
interest groups, and the press. Problems contributing to inadequate service 
to users were discussed. 
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Goodman, Arnold P., and others. POD User-Needs Study, Phase II — Flow of 

Scientific and Technical Information Within the Defense Industry. Final 
Report . Anaheim, Calif. j North American Aviation, Inc., November 1966 
(Avail. ERIC ED 018-239 and ED 018-240; Overview ED 018-237). 

A study was made of the nation's defense industry to determine its infor- 
mation needs and the flow of scientific and technical information (flow process) 
necessary to satisfy these needs. Volume I of the three volume final report 
was an overview of the other two volumes and included a nontechnical summary, 
guidelines for management decisions, and recommendations for the future. Vol- 
ume II described the technical approach, findings, and recommendations of the 
study. The third volume presented the reduced data collected frc»n 1500 indi- 
viduals in 83 organizations within the defense industry. Analysis of the data 
included frequency distributions, stepwise regression relationships, and com- 
parison of phase I (an earlier part of the study carried on within the defense 
industry) and phase II distributions. 

Grose, J. H., and Krall, George F., Jr. The Conceptual Framework for a Ccanpre - 
hensive and Continuing STl Resource Inventory . Radnor, Pa.: Krall Manage- 

ment, Inc., April 1978 (Prepared for National Science Foundation; Avail. 
NTIS PB 282-345) . 

An assessment of the requirements of users and suppliers of these resources 
and the current STI enterprise capabilities was made. Various implementation 
considerations were ccsipiled and synthesized, and a conceptual framework for 
the continuing pursuit of an STI resource inventory was formulated. The pri- 
mary function of such an STI resource inventory was to act as an Interface 
between the users and suppliers of STI in the information industry. 

Hall, Homer J. Oser Values in the Selection of Information Services . Linden, 
N.J.: Exxon Research and Engineering Co., February 1977 (Prepared for 

National Science Foundation; Avail. NTIS PB 269-507). 

The value systems by which the users of purchased information services 
select which one to use or buy were found to differ sharply between different 
user populations. Selection variables were summarized in a matrix of user 
values and interaction effects. The research function user preferred original 
documents and tended to choose subjectively. The operating manager more often 
stopped with abstracts and emphasized quantifiable factors. The planning 
manager wanted a digest of ideas, not references, and tended to judge a service 
first by its reliability. These three functions attached entirely different 
connotations to the same words for key parameters. Apparent conflicts could be 
resolved by dimensional analysis of differences in values perceived as illus- 
trated by case histories, interviews, and questionnaire results. 



Hall, Homer J. "Which Service to Buy and Why?" Information Politics. Proceed - 
ings of the 39th Annual Meeting of the American Society for Information 
Science . Edited by Susan K. Martin. Washington, D.C.j American Society 
for Information Science, 1976. 

Identifiable groups of users of scientific and technical information ser- 
vices were found to differ sharply in the value systems they applied in decid- 
ing what service to buy or use. A guide to user-identification developed for 
NSP/DSI took the form of a dimensional analysis of characteristic selection fac- 
tors and a matrix of their interaction effects, when selection factors were 
grouped into three dimensions (quantifiable, qualitative, and judgmental), dif- 
ferences were observed in the relative importance attached to these by differ- 
ent user populations. Value systems varied with the specific use, and use 
appeared to be a more significant factor than the user as an individual. 

Herner, Saul. "Determination of User Needs for the Design of Information Sys- 
tems." In Information Systems Workshop: The Designer's Res pons i bill tv 

and His Methodology . Washington, D.C.: Spartan Books, 1962. 

For designing information storage and retrieval systems, this study advo- 
cated compromise to achieve broad user acceptance, maximum economy, and minimum 
bias. User requirements were said to be Important not only in determining what 
should be included in a system, but also what is not needed, and what is needed 
but available from other sources in a satisfactory manner. Analytical tech- 
niques for measuring relevance to user needs and sources of bias were discussed. 

Herner, Saul, and Herner, Mary. "Information Needs and Uses in Science and 

Technology." in (1967) Annual Review of Informat ion Science and Technol- 
ogy. Vol. II. Edited by Carlos A. Caudra. New York; John Wiley and 
Sons, 1 967, 1-21 . 

This review primarily covered publications appearing in 1966. The publica- 
tions were discussed in categories based on methodology employed. Sections 
included the present state and problems of user studies, diaries and user- 
administered records, interviews observations, questionnaires, indirect studies, 
combined technique studies, and implications and conclusions. 
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Harmon, Glynn. "Information Need Transformation During Inquiry: A Reinterpre- 

tation of User Relevance." The Information Conscious Society. Proceedings 
of the 33rd Annual Meeting of the American Society for Information Science . 
Edited by Jeanne B. North. Washington, D.C.s American Society for Infor- 
mation Science, 1970, 41-43. 

Findings of two key relevance investigations, the Cuadra-Katter and Rees- 
Schultz experiments, were reinterpreted against a user information need model. 
Nonexpert users, like initiators of research, generally possessed few if any ade- 
quate cognitive organizers. They must mentally acquire and permute information 
elements until a personally satisfactory conceptual organization emerges. Expert 
users at the outset of inquiry, or researchers in the post-hypothesis stages of 
inquiry, generally possessed adequate cognitive organizers. As the user elabo- 
rated his cognitive map, information gaps and alternative arrays became increas- 
ingly apparent. He was then better able to designate various information ele- 
ments as either essential, redundant, or irrelevant to his cognitive mapping. 

The system user thus needed alernative access to both organizing form and 
content . 

Kidd, J. S., and others. "Information Seeking Behavior of Scientists as a Guide 
to Information Product Design." Current Research on Scientific and Techni- 
cal Information Transfer . N.Y.: Jeffrey Norton Publishers, Inc., 1977. 

Specific instances of user need assessment and information product develop- 
ment were described. The work reported was one component of a multiphase pro- 
ject. The particular user set studied in this phase was comprised of scientists 
engaged in cross-disciplinary projects, mainly environmental studies. An infor- 
mation product consisting primarily of extracted and reformatted journal mater- 
ials in the topic area of the effect of herbicides on aquatic plants was employed 
to illustrate the outcomes. 

King, Donald W. , and Palmour, Vernon E. "User Behavior." Changing Patterns in 
Information Retrieval, Tenth Annual National Information Retrieval Collo - 
quium . Edited by Carol Penichel. Washington, D.C.: American Society for 

Information Science, 1974, 7-33. 

I 

The paper discussed a conceptual framework for user behavior, the knowledge 
gained concerning the user interface with the system, and what more needed to be 
discovered about system users and their behavior. Since this Colloquium was 
traditionally focused on formal scientific and technical information systems, 
the paper was limited to this area. Discussed but purposely not stressed, were 
informal communications within organizations and invisible colleges, although 
the importance of the informal mechanisms for transmitting information was not 
underestimated. 



Kunz, Werner, and others. Methods of Analysis and Evaluation of Information 
Needs . Munich, W. Germany: UNBSCX), 1 977 (Avail. Verlag Dokumentation, 

Postfach 711009, 8000 Munich 71, West Germany). 

Trends and approaches of user studies in the information sciences were 
reviewed, leading to strategies for future work. The study analyzed and eval- 
uated information needs, examined their practicability and utility within the 
framework of planning national information systems (i.e., planning the national 
documentation, library and archive infrastructures) and proposed a direction 
for future user and use research. It was organized in six parts: introduction, 

theoretical concepts of information needs and behavior, conventional empirical 
methods of user research, newer approaches and methods for analyzing and eval- 
uating information needs, conclusions, and prospects. A list of 124 references 
completed the work. 

Ladendorf, Janice M. "Information Plow in Science, Technology, and Ccxnmerce." 
Special Libraries . XV (May/June 1 970) , 21 5-221 . 

Information of user characteristics has been of crucial importance in the 
design of information systems. Unfortunately, many scientists and engineers 
have not used information even when it was readily available to them. Charac- 
teristic communication behavior was established for two basic types of users: 
the successful research scientist and the average Industrial technologist. Com- 
munication activities did not represent an isolated behavior pattern. They were 
deeply tied to social, professional, and institutional relationships. There 
were limits on the extent to which these behavior patterns could be modified 
to increase the use of information. 

Lancaster, P. Wilfrid. "Assessment of the Technical Information Requirement of 
Users," in Contemporary Problems in Technical and Library Information 
Center Management . Edited by Alan Rees. Washington, D.C.: American 

Society for Information Science, 1974, 59-85. 

Measurement of technical information needs of users was discussed, and the 
various types of user surveys, which could be categorized as library-oriented 
and user-oriented, were described. Methodologies of the surveys were discussed 
and evaluated as to effectiveness. General conclusions which could be drawn 
fr<xn user studies were: libraries and information centers, as well as librar- 

ians, were not ranked highly as information sources; librarians may be respons- 
ible for this problem in their failure to educate their patrons and themselves; 
information sources were generally chosen based on ease of use; personal collec- 
tions were preferred to even the finest central information facility; and infor- 
mal channels of communication, such as personal contacts, were frequently more 
highly valued than formal channels. The applicability of user studies to Army 
and other government information centers was discussed. 



Landau, Herbert B. "Methodology of a Technical Information Use Study." Special 
Libraries , LX (July/August 1969), 340-346. 

A methodology for conducting a study of the needs of a technological infor- 
mation user was presented. The methodology could be used, with little or no 
modification, by an organization of moderate size where the majority of the 
technical staff members were situated at one main location. This technique util- 
ized an informal interview approach which could be employed by a single inter- 
viewer. The technique was employed at the Auerbach Corporation in the first 
half of 1968 and was found to be successful. 

Lin, Nan, and Garvey, William D. "Information Needs and Uses.” In (1972) 

Annual Review of Information Science and Technology , Vol. VII. Edited by 
Carlos A. Cuadra. Washington, D.C.: American Society for Information 

Science, 1972, 5-37. 

This publication was a summary of the 1971 literature on (1) attitudinal and 
behavlorial patterns of information needs, exchange, and uses among scientists 
and technologists, and (2) organizational and institutional efforts to manage 
and improve the information system. Topics discussed were: factors generating 

differential Information needs; types of information needs; patterns of informa- 
tion exchange; information uses of scientists, technologists, and laymen; and 
innovations in scientific and technical information systems. (96 references) 

Lipetz, Ben-Ami. "Information Needs and Uses." In (1970) Annual Review of 

Information Science and Technology . Vol. V. Edited by Carlos A. Cuadra. 
Chicago: Encyclopedia Britannica, Inc., 1970, 3-32. 

This annual review covering selected 1 969 literature contained three major 
sections; surveys and measurements, methodology, and theory. The surveys and 
measur«nents section contained reviews of publications that reported actual data 
on information uses and needs. The methodology section covered publications 
that reported on new techniques for observing or detecting information uses and 
needs. The third theory section discussed articles on definition, explanation, 
prediction, and theory. (114 references) 

Martyn, John. "Information Needs and Uses." in (1974) Annual Review of Infor- 
mation Science and Technology . Vol, IX. Edited by Carlos A. Cuadra. 
Washington, D.C.: American Society for Information Science, 1974, 3-24. 

The author reviewed selected publications issued during 1973 and part of 
1972 dealing with Information needs and uses. Major categories of references 
discussed were systeras-oriented studies, canponent-oriented studies, and back- 
ground research. (32 references) 



Menzel/ Herbert. "Information Needs and Uses in Science and Technology." In 
(1966) Annual Review of Information Science and Technology . Vol. I. 

Edited by Carlos A. Cuadra. New York: John Wiley and Sons, 1966. 

For this first volume of the annual Review, coverage included studies pub- 
lished as early as December 1963. The reader was given a rounded picture of a 
period during which research progressed along the following lines: (V) sub- 

stantial advances in the more systematic and fruitful application of the (crit- 
ical incidents) technique, (2) the introduction of at least two very promising 
approaches (solution development records and comparison of research teams work- 
ing on identical tasks) , and (3) an increase in the number of efforts at compre- 
hensive study of the information-flow situation in given disciplines. Selected 
papers published during this period were grouped and reviewed under the follow- 
ing classifications: preference, demand, and experimental studies; use studies 

(e.g., channel studies and critical incident studies); dissemination studies; 
and comprehensive study programs. 

Paisley, William J. "Information Needs and Uses." In (1968) Annual Review of 
Information Science and Technology . Vol. III. Edited by Carlos A. Cuadra. 
Chicago: Encyclopedia Britannica, 1968, 1-30. 

In this review of 1967 literature, the author examined studies of 
information-gathering amd information-disseminating behavior of scientists and 
technologists. The publications were reviewed in the conceptual framework of 
the scientist within a series of systems: his own head, his work team, a formal 

organization, cui invisible college, his reference group, a membership group, a 
formal information system, a political system, and his culture. (68 references) 

Ranganathan, S. R. "Annotation on 'User's Survey.'" Annals of Library Science 
and Documentation , (March/June 1970), 1-22. 

Substantial comments by Ranganathan on a paper, "User's Survey Concerning 
Teachers and Research Scholars in the Department of Chemistry, University of 
Delhi," by Krlshan Kumar (ISA 70-3030) were presented. Direct quotes from 
Kumar's paper were followed by Ranganathan ' s comments. The author offered sev- 
eral suggestions for the improvement of survey reports, including unity of idea, 
sequence, verbal plane, section heading, notational plane, index and cross ref- 
erence, and technical terminology. 



Rawdins, Eugene., "Field Survey of Information Needs of Industry Scientific and 
Technical Library Users." Information Revolution. Proceedings of the 38th 
Annual Meeting of the American Society for Information Science . Edited by 
Charles W. Husbands. Washingtonf D.C.: American Society of Information 

Sciencef 1975, 41-42. 

Library resources may be regarded as products and services. Library users 
in industry exchange time and either direct or allocated funds when using 
library resources. That is, a library visit involves a transaction. Whenever 
a transaction occurs, consumer behavioral theory may be invoked. Consumer 
behavioral theory and related information science literature formed the basis 
for selecting variables for a field study of industrial scientific and techni- 
cal library users. One of the purposes of the study was to investigate and 
relate information need/use parameters to other parameters in order to describe 
and explain the behavior of industry scientific and technical library users. 

Rubenstein, Albert H. , aind others. Critical Field Experiments on Uses of Scien- 
tific and Technical Information . Evanston, 111.: Northwestern University, 

November 1977 (Prepared for National Science Foundation; Avail. NTIS 
PB 279-693) . 

A two year project was designed to identify critical field experiments 
involving behavior of users of scientific and technical information (STI) . 

Tasks of the project included; literature analysis, selective surveys of tar- 
get audiences; developnent of a propositional inventory and an overall frame- 
work; and delineation of researchable questions and specific experimental 
designs. The output included field experimental designs on these subjects: 
the effects of accessibility, ease of use, and quality on preference for and use 
of STI systems and services (SS) ; the roles of key communicators; group cohe- 
siveness as a factor in adoption of new STI/SS; marketing strategies for STI in 
LDCs; the role of face-to-face communication; nonstandard search strategies; 
and the integrated nature of Technology Exchange Transactions. 

Rubenstein, Albert H., and others. Critical Field Experiments on Uses of Scien - 
tific and Technical Information . Evanston, 111.: Northwestern University, 

Dept, of Industrial Engineering and Management Sciences, November 1975 
(Avail. ERIC ED 1 43-302) . 

Research was done on the behavior and preferences of researchers with 
respect to technical literature, computer-based information systems, and other 
scientific and technical information (STI) systems and services. The objec- 
tives on this project were: (1) to review the state-of-the-art in various 

aspects of the field and to develop researchable questions, and (2) to design 
key experiments to be carried out in a large number of organizations and 
administrative experiments to be carried out by individual managers of research 
and development (R&D) or STI to solve specific STI problems. Other major sec- 
tions of the report included a literature analysis, a survey of STI users, a 
survey of on-going and proposed experiments by STI users, the development of a 
propositional inventory, and the exploration of the feasibility of a framework 
for analysis of the STI process. A list of working papers on the same project 
were presented. 



Schoenborn, Theodore P. "Information User Needs Survey — A Decision Guide for 
Resource Allocation." The Information Age in Perspective. Proceedings of 
the 41 St Annual Meeting of the American Society for Information Science . 
White Plains, New York: Knowledge Industry Publications, Inc., 1978, 

287-290. 

The value of an information user needs survey in determining appropriate 
service levels was established: concrete information. Instead of the expecta- 

tions of the management, would be made available. In addition, NIOSH program 
objectives were in the context of the staff's perceived need for and satisfac- 
tion with the information services and resources available. 

Schuchman, Hedvah L. Information Transfer in Engineering . Glastonbury, Conn.: 
The Futures Group 1980 (Prepared for the National Science Pounation; Avail. 
The Futures Group, 76 Eastern Blvd., Glastonbury, Ct. 06033). 

The effects of changes in scientific and technical information transfer on 
the engineering profession were examined. The objectives of the study were to: 
identify the information used by technologists of various disciplines and spe- 
cialties, identify the major modes of communication and patterns of use, 
describe changes which might be anticipated in information use patterns and 
communication modes as a result of changes in information storage and dissemi- 
nation methods, and identify policy implications resulting fran these evolu- 
tionary changes. 

By means of a large scale survey, a broad spectrum of engineers were asked 
to provide information on their use of scientific and technical information. 

Data collected were analyzed to provide information profile and use patterns of 
engineers and to project changes in the profile and use pattern resulting from 
advances in information processing technologies. 

Skelton, Barbara. "Scientists and Social Scientists as Information Users; A 
Comparison of Results of Science User Studies with the Investigation Into 
Information Requirements of the Social Sciences." Journal of Librarian- 
ship, V (April 1973), 138-1 56. 

An attempt to compare the information seeking behavior of scientists and 
social scientists was made. A general profile of the scientist as an informa- 
tion user was drawn, based on thirteen science user studies which contained 
data for areas covered by the INFROSS (Investigation Into Information Require- 
ments of the Social Sciences) survey. A profile for the social scientist as 
information user was drawn, based on the results of a national survey of a wide 
range of social scientists in different disciplines and environments. It was 
concluded that scientists and social scientists did not differ to any large 
extent. in their information seeking behavior. However, the extent of use of 
different methods and the degrees of problems encountered were varied and 
accounted for differences in the information seeking behavior of scientists and 
social scientists. 



Skelton, Barbara. "Investigation Into Information Requirements of the Social 
Sci ences . " Design of Information Systems in the Social Sciences: Working 

Paper I . University Library, Bath University of Technology, U.K. , July 
1971 (Avail. ERIC ED 078-866). 

The investigation into Information Requirements of the Social Sciences 
(INFROSS) was conducted for the Office of Scientific and Technical Information 
from 1968 to 1970. The study was on a national scale and sought to obtain data 
on information requirements of social scientists in various environments. This 
report attempted to draw from previous science user studies possible general 
conclusions that could be ccmpared with the results of INFROSS. The investiga- 
tors concluded that because there were many problems connected with comparing 
studies, only broad trends could be discerned. It was concluded that informa- 
tion requirements varied according to job function and scientific discipline, 
that user studies had been poorly designed and could not be correlated, and 
that better studies were needed. A review and comparison of the findings of 
previous science user studies and a description of problems encountered in com- 
paring studies were included. 

Thuronyi, Geza T., and Pietkiewicz, Wesley. "User Participation in an Informa- 
tion System. " The Information Conscious Society. Proceedings of the 33rd 
Annual Meeting of the American Society for Information Science . Edited by 
Jeanne B. North. Washington, D.C.; American Society for Information 
Science, 1 970, 1 41-146. 

A small information system was described from three points of view: as 

a bibliographic support service to the sponsoring agency, as a published index 
for general dissemination, and as part of a large information system. It was 
shown how the principal users (research scientists of the sponsoring agency) 
were built into the system and how the feedback received from them improved the 
system, especially in the areas of document selection, indexing, and biblio- 
graphic format. It was proposed that large information systems could be con- 
ceived in the form of interlocking cells, each of which incorporated a repre- 
sentative user group as part of its regular activity. 

Tierney, Cliff. "Users' Role in the Development of a Technical Information Net- 
work Serving a Decentralized Manufacturing Company." Innovative Develop - 
ment in Information Systems; Their Benefits and Costs. Proceedings of 
the 36th Annual Meeting of the American Society for Information Science . 
Edited by Helen J. Waldron and F. Raymond Long. Washington, D.C.: Ameri- 

can Society for Information Science, 1973, 229-230. 

The effectiveness of technical communications in industry was found to 
depend more on the system used than on the attitude of those using the system. 
Whirlpool Corporation approached the development of a technical information 
transfer system in terms of user preferences and attitudes and found that the 
process of development produced the key benefit. 
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Wood, Daniel N. "User Studies: A Review of the Literature Prom 1966-1970." 

Aslib Proceedings , XXIII (1971), 11-23. 

The paper considered a wide range of investigations into the information 
gathering habits of scientists, engineers, social scientists, and others and 
reported information on which managers could base policies regarding library 
and information services at all levels. Although an increasing number of 
studies were carried out in Eastern Europe and in the USSR, no major work was 
reported. Consequently, the review considered mainly British and American 
Investigations. 

Young, Mary E. User Needs in Documentation and Information . II. Springfield, 

Va.: National Technical Information Service, April 1979 (Avail. NTIS 

PB 80-807910). 

Reports on meeting the needs of the Information sciences user in the most 
direct and specific way were cited, including methods used to determine those 
needs and measures taken to fulfill them in educational resources, data bases, 
data management, library networks, photographic recording systems, technical 
information centers, and consulting services. This updated bibliography con- 
tained 213 abstracts, 50 of which were new entries to the previous edition. 



INFORMATION - ECONOMICS 

Bickner, R. E. "Goncepts on Economic Cost." In (1971) Cost Consideration in 
Systems Analysis . Edited by Gene H. Fisher, N.Y.: American Elsexier 

Publishing Company, 1970, 24-63. 

Evaluation and design of information systems cannot be performed without 
scMne unifying concept of economic costs. In order to set forth a general frame- 
work for economic costs, this paper was included in the key papers. The 
examples presented in the paper were not related to information science, 
although they were easily transferable. The paper presented a very clear, under- 
standable and non-mathematical explanation of a broad range of issues concerning 
economic costs. These Issues included such important concepts as cost versus 
benefits; how to identify, measure and evaluate costs; how to handle past and 
future costs; minimizing costs versus maximizing benefits; the distinction 
between dollar expenditures and total cost; discount theory; macro-cost analysis 
versus micro-cost analysis and so on. The paper was highly useful to the novice 
as well as to persons engaged in cost analysis. 

Bishop, Ethelyn P., and Clayton, Audrey. "An Application of Market Research 
Techniques to the Dissemination of Scientific and Technical Information." 

The Value of Information. Collection of Papers Presented at the 6th Mid - 
Year Meeting, Americal Society for Information Science . Syracuse 
University, Syracuse, N.Y. ; American Society for Information Science, 

May 1977, 12-16. 

A methodology which would improve user access to scientific and technical 
information (STI) by enabling producers and distributors of STI to better target 
their product design, distribution, and promotion efforts was described. An 
analysis of the data gathered through a pilot survey of 402 electrical engineers 
and chemists indentified six segments in each discipline whose members had 
similar STI needs and other common characteristics including demographics, 
sources used to satisfy STI needs, and perceptions of current STI sources. 

Bishop, Ethelyn F., and Misenoff, Norman. An Application of Market Research 
Techniques to the Dissemination of Scientific and Technical Information . 
Arlington, Va.: Forecasting International Ltd., February 1977 (Prepared 

for National Science Foundation; Avail. NTIS PB 265-130). 

This study used the market research technique of market segmentation and 
cluster analysis to identify segments of the user market with similar STI needs 
and to describe these groups in terms of their common STI use characteristics, 
their personal characteristics, and their access requirements, thus allowing 
the development of strategies more responsive to those needs. A questionnaire 
was administered to 201 electrical engineers and 201 chemists in person inter- 
views. Each respondent was asked to rank types of scientific and technical 
information in order to need. These rankings were submitted to a computerized 
statistical analysis. Six segments were identified within each discipline 
included in the sample. 
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Cooper, Michael D. "A Cost Model for Evaluating Information Retrieval Systems." 
Journal of the American Society for Information Science , XXIII (September/ 
October 1972), 306-312. 

A mathematical model for use in studying how to minimize the cost of 
operating a mechanized retrieval system was developed. Through the use of cost 
analysis, the model provided a method for comparative evaluation of information 
retrieval systems. 

Cooper, Michael D. "Evaluation of Information Retrieval Systems: A Simulation 

and Cost Approach." Unpublished Ph.D. dissertation. University of Cali- 
fornia, Berkeley, 1971 (Avail. University Microfilms 71-25445). 

Two specific approaches to evaluation of information retrieval systems were 
explored. The first is a mathematical model for use in studying how to mini- 
mize the cost of operating a mechanized retrieval system. This model divided 
the costs of a retrieval system into two components: syst®n costs and user 

costs. With this approach it was possible to determine the allocation of user 
and system time that minimized the total cost of operating the system. The 
second approach to the evaluation of literature searching systems was the 
development of a simulation model as a preliminary step toward the creation of 
a tool for system design and evaluation. The simulation program created a well 
specified collection of documents and analyzed the effect of changes in query 
file characteristics on system performance. 

King, Donald W. , Roderer, Nancy K., and Olsen, Harold A., eds. Key Papers in 
the Economics of Information . White Plains, N.Y. : Knowledge Industry 

Publications, Inc., 1980. 

In its three major parts: Economic Cost of Information, Pricing of Infor- 

mation, and Value of Information, this volume provided a ccxnprehensive look at 
an issue of timely importance to library and Information science professionals. 
Both theoretical and applied viewpoints were represented, from a general dis- 
cussion of values and benefits to the very basic and controversial question of 
whether a public library should charge for providing infomation. 

Lancaster, F. Wilfrid. "The Cost-Effectiveness Analysis of Information 

Retrieval and Dissemination Systerms." Journal of the American Society 
for Information Science . XXII (1) (January/Pebruary 1 971 ), 1 2-27. 

A distinction was made between cost-effectiveness analysis and cost- 
benefits analysis as applied to information systems; and the relationship 
between costs, performance, and benefits was discussed. Scmie factors influenc- 
ing the cost-effectiveness of retrieval and dissemination systems was identified. 
Various- aspects of system operation that were susceptible to cost-effectiveness 
analysis were discussed, including system coverage, indexing policies and pro- 
cedures, system vocabulary, searching procedures, and mode of interaction 
between system and user. Possible tradeoffs between input and output costs, 
and the effects of these tradeoffs on cost-effectiveness were presented. 



Ljungberg, Slxten# and Tullgren, Ake. "Investigation on the Use of Information 
in R&D in a Research Intensive Company." In (1977) Information Management 
in the 1980's. Proceedings of the 40th Annual Meeting of the American 
Society for Information Science . White PlainSf N.Y.: Knowledge Industry 

Publications, Inc . , 1 977 . 

The objective of this case study of a pharmaceutical firm was survey the 
balance between the users' well-defined needs for information, especially with 
regard to current awareness, and the allocation of resources. The study was 
carried out through inquiries, interviews, quantitative measurements of the 
handling of information, and qualitative appraisals of sourcs ans selected 
material. The results were evaluated with regard to organizational aspects and 
from the point of view of technical communication. Great attention was paid to 
cost-benefit analyses and to methods for continuously checking the effective- 
ness and the profitability of the Information and Documentation Unit. Such 
operations, in combination with interviews, have turned out to provide a good 
basis for budgeting and strategic planning. 

Olsen, Harold Anker. "The Economics of Information: Bibliography and Com- 

mentary on the Literature." Information Part 2 Reports and 
Bibliographies , 1 (2) (March/April 1972), 1-40, (Avail. ERIC ED 076-214) 

Since the date of original publication, nearly 500 items were located which 
pertained to the economics of information. The 160 deemed roost significant 
were selected for Inclusion in the supplementary listing. As in the first 
edition, citations were included from both the information literature and the 
economics literature. Perhaps the most significant trend noted this year was 
the substantially greater awareness by the information profession of the sig- 
nificance of economic factors to information management. 

Olsen, Hcurold Anker. The Economics of Information: Bibliography and Ccanmentary 

on the Literature . Washington, D.C.: ERIC Clearinghouse on Library and 

Information Science, January 1971, (Avail. ERIC ED 044-545). 

This report explored the bodies of literature pertinent to the economics 
of information, a topic of growing interest to the infomation community and to 
economists. As used here, economics of information referred to the concepts 
and tools of economics as they applied to information activities. The report 
consisted of (1) a short section of the economists' framework for analysis, 

(2) a table that divided the pertinent literature into 14 categories, briefly 
defined each category, and explained why it was important to information 
activities; (3) a brief commentary on the state of this literature; and (4) a 
selected bibliography of more than 300 items. An exhaustive list of all items 
was not done in this initial effort, but most recent literature, especially 
monographs, technical reports, and literature surveys, were covered to the 
extent that a reader could obtain a good introduction to this literature. In 
addition, 25 items were identified as giving particularly informative over- 
views. These items represented the full range of material from theoretical 
studies to applied analyses, plus several surveys. 



Taylor, Robert S., ed. Eoonomics of Information Dissemination; A Symposium . 
Appearing in Frontiers of Librarianship, XVI, Syracuse, N.Y.: Syracuse 

University School of Library Science, 1973 (Avail. Syracuse University 
School of Library Science) . 

A symposium was held to explore the problems of cost and benefits of 
information and its dissemination. Topics covered in the symposium included: 

(1) the development of a concept of information in the marketplace as an 
economic commodity; (2) the role of information in the decision-making pro- 
cesses; and (3) viable models for management decision-making in support of 
planning cost-benefit considerations, long-term budgeting; and (4) the design, 
development, eind marketing of information products. 

Tressel, George W. , and Brown, Patricia L. A Critical Review of Research 
Related to the Economics of the Scientific and Technical Information 
Industry. Final Report . Columbus, Ohio: Battelle Memorial Institute, 

March 1975 (Avail. ERIC ED 111-429 and NTIS PB 245-665). 

A review of expert opinion and recent literature on the critical issues 
and open questions relating to the economics of the scientific and technical 
information industry was presented. The authors concluded that: (1) there was 

hardly an area that did not call for more and better research, not nearly so 
much for quantity as for a more coherent and meaningful pattern, (2) a broad 
effort was needed to collect available data and to develop standardized collec- 
tion methodologies which could meet critical scrutiny as well as provide the 
necessary comparative and additive information, and (3) increased research and 
policy study of the roles and processes of information diffusion into the 
private sector was also needed. 

Zais, Harriet W. "Economic Modeling; An Aid to the Pricing of Information 

Services." Journal of the American Society for Information Science , XXII (2) 
(March 1 977) , 89-95. 

As budgets tightened in the late 1960's and early 1970's, many policy- 
makers and administrators began to ask whether users of information products 
and services should be charged for their use. Several studies were conducted 
at that time to look into methods for charging. The article presented a good 
summary of the methods that could be used in pricing information products and 
services. Typically, price strategies of these products and services involved 
(1) service provided at no charge, (2) average cost pricing, (3) price discrim- 
ination, or (4) marginal cost pricing. This article discussed the strengths 
and weaknesses of each of these pricing strategies. Furthermore, an example 
was presented which served as a prototype for pricing selective dissemination 
of information (SDI) products and services. 
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APPENDIX 


A PROJECT PI^ FOR THE REVIEW AND EVALUATION OF THE 
LANGLEY RESEARCH CENTER'S SCIENTIFIC AND TECHNICAL INFORMATION PROGRAM 


INTRODUCTION 

One of the most important results of exploration and research and develop- 
ment is information. The National Aeronautics and Space Administration's 
scientific and technical information system is one of the largest and best 
known federal STI programs in the country. The mission of the NASA STI is two- 
fold: (1 ) to acquire worldwide research in aeronauticSf space* and related dis- 

ciplines to keep NASA personnel abreast of current activities and developments; 
and (2) to contribute to the expansion of STI through timely dissemination of 
NASA-generated and -sponsored research, development, testing, and technical 
evaluations. The Langley STI program is an integral part of the Agency's STI 
program and is responsible for implementing Agency and Center policies concern- 
ing the management of STI. Expeditious publication of the Center's research 
is Langley's contribution to the Agency's goal of timely dissemination of 
NASA research. 


BACKGROUND 

The Langley Research Center (LaRC) is one of the leading national labora- 
tories for research and development in the sciences of aeronautics and space 
technology. Founded in 1917, Langley was the nucleus of the former National 
Advisory Committee for Aeronautics (NACA) . For more than 60 years, Langley 
engineers, scientists, and technicians have been conducting basic and applied 
research in fluid and flight mechanics, flight systems, structures and materials, 
acoustics and noise reduction, measurements and instrumentation systems, data 
systems, and space and earth sciences. The results of this research are 
disseminated through NASA scientific and technical publications as well as non- 
NASA media such as technical or professional society journals and similar 
periodicals; dmestic and foreign presentations of papers, talks, and lectures; 
and in the proceedings of conferences and symposia. For calendar year 1979, the 
output of the Center's 1,330 Aerospace Technologists (AST's) totaled 1,061 items 
which included 186 NASA formal technical publications; 116 Quick-Release 
Technical Memorandums; 149 journal articles; and 610 speeches, lectures, and 
presentations. 

The documented research output of the Center is processed throughout the 
Langley Scientific and Technical Information Programs Division (STIPD), which is 
an integral part of the Agency's scientific and technical information program. 


STATEMENT OF THE PROBLEM 

During the 63-year history of the Langley Research Center, a comprehensive 
review and evaluation of the Center's STI program has never been conducted. 
Portions of the Center's STI program have received periodic or occasional 
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assessment; however, no valid empirical data exist which can be used to 
evaluate the total progreun's efficiency and effectiveness. 


PURPOSE OF THE STUDY 

A comprehensive review/evaluation of the Center's STI program will seek to 
determine the extent to which the program is meeting the needs of Langley 
research and professional personnel and the recipients of Langley-generated 
scientific and technical information, the areas or portions of the program which 
need improvement, and ways in which the program can be modified to improve its 
overall efficiency and effectiveness. In conjunction with the evaluation proj- 
ect, a theoretical and analytical review of the NASA formal report as a medium 
for information transmittal will be conducted. The results of the study will 
enable NASA to develop a more effective medium for transmitting the results of 
its research. 

An annotated bibliography of literature citations on the topic of the 
design and evaluation of scientific and technical information systems will 
be ccanpleted and published as a resource for future evaluations. 


t 


w 


significance 

This study will provide information which can be used to evaluate and 
improve the Langley STI program. The information gathered by this study will 
establish the following: 

1 . Knowledge of and attitudes toward the Langley STI program by internal 
and external users 

2. Information needs of internal and external users of Langley STI 

3. Perceived usability, technical quality, and prestige of Langley formal 
series reports eind journal articles by these users 

4. Familiarity, use of, and attitudes toward selected NASA STI products and 
services by these users 

5. Assessment of the services provided by STIPD by Langley researchers, 
identifying areas of concern and recommendations for improvement 

6 . Recommendations for improving the effectiveness of the dissemination of 
Langley STI 

7. Effectiveness of the Center's policies and procedures for managing and 
publishing Langley STI 

8. Bibliography of literature citations on the topics of STI transfer and 
dissemination models, systems, and procedures 
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9. Bibliography of literature citations on the evaluation of the STI pro- 
grams/ and 

10. Effectiveness of the NASA Formal Report as a medium for transmitting 
information. 


Overview 

The study will utilize both descriptive and experimental research and will 
be directed by Thomas E. Pinelli, Assistant Chief/ STIPD. A steering committee 
of 1 7 individuals will be used to help focuS/ develop/ and guide the study 
through its completion. Each research division will nominate a representative 
to serve on the committee. Mr. George Chandler/ Chief, Scientific and Technical 
Information (STI) Branch, NASA Headquarters, will serve as an ex-officio member 
of the committee. The individual tasks established for the study will be 
executed using Iinmgley, Old Dominion University, and professional contract 
personnel. 


Limitations 

The study will be limited to the scientific and technical information out- 
put of the Center as processed or disseminated through the Langley STI program. 
The study is not concerned with either informal transfer or secondary applica- 
tion of the Center's research output. The study will involve researchers at the 
Langley Research Center and NASA information users in other government agencies, 
industry, and academic institutions. 


REVIEW OP RELEVANT RESEARCH 

A search is underway to identify literature relevant to the study. The 
results of Langley and Headquarters' STI studies and assessments conducted since 
1 968 will be collected and used to help develop the research methodology for the 
study. A review of existing systems and models for transferring and disseminat- 
ing scientific and technical Information and evaluating scientific and technical 
information programs will be undertaken. In addition, an annotated bibliography 
of literature citations on the topics of the transfer, dissemination, and 
evaluation of scientific and technical information programs will be completed. 


RESEARCH METHODOLOGY 

• The study will investigate the effectiveness and efficiency of the Center's 

scientific and technical information program, with particular emphasis placed on 
improving the effectiveness of the dissemination process. The specific actions 
" to be taken are described in the following phases. 
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Phase l! Knowledge and Attitudes Survey, Langley Rese arch Personnel 

Phase I of the review and evaluation project requires an assessment of the 

adequacy of the Center's STI program in meeting the needs of Langley research 
and professional personnel. Areas of the program which need improvement will 
be identified and ways in which the program can be made more effective will be 
recommended. This task involves (1) determining through open-ended questions 
during in-depth interviews the areas and dimensions of the program which 
researchers consider important, (2) constructing a closed-ended survey to be 
distributed to all research personnel, (3) tabulating and analyzing the 
responses to the closed-ended questions and conpiling and analyzing the pro- 
posed changes and recommendations solicited by several open-ended questions 
and, (4) presenting the findings of the questionnaire in a final report. 

The outcome of Phase I will be an evaluation of Langley's and the Agency's 

programs for meeting the needs of Langley research and professional personnel. 


Phase II: Audit of Publication Process 

Phase II of the review and evaluations project requires an "audit" or 
management analysis of the policies, procedures, and practices used by the 
Langley Research Center to process, publish, or otherwise handle scientific and 
technical information. This task involves (1) identifying the various media used 
by the Center to output its scientific and technical information; (2) compiling 
all regulations, policies, and instructions applicable to these media; (3) 
documenting the procedures as currently prescribed; (4) canparing current or 
actual practices with published management instructions to identify discrep- 
ancies or gaps in procedural guidance; and (5) recommending additional or modi- 
fied procedures. 

The outcome or stated purpose of the task is to define the total current 
procedural framework for processing, publishing, or otherwise handling Langley's 
scientific information and to supplement existing practices and procedures to 
create a comprehensive, effective, understandable, and practical framework 
covering the handling of all research output. 


Phase Hit Audit of the Report and Manuscript Control Office (RAMCO) 

Phase III of the review and evaluation project requires an audit or 
management analysis of the policies, procedures, and practices used by RAMCO 
(Report and Manuscript Control Office) to manage and report the Center's 
scientific and technical information output. 

The audit involves (1) documenting the current manual system using flow- 
charts, tables, and other systems analysis tools and techniques; (2) determining 
whether changes to the current manual system are necessary and justifiable; 

(3) proposing a new manual or automated (internal or external) system with 
appropriate justification for selection; (4) examining the feasibility of 
in-house autanation capabilities; and (5) presenting the procedural framework, 
underlying models, analysis, comments, and recommendations in a final report. 
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The outcome or stated purpose of the audit will be an analysis and docu- 
mentation of the current RAMCX) operations, identifying areas for potential 
improvement including possible automation. The audit will emphasize the records 
management aspect of the operation. 


Phase IV; Knowledge and Attitudes Survey, Industrial and Academic Personnel 

Phase IV of the review and evaluation project requires an assessment of the 
benefits, usage, and perceived quality of the NASA/Langley STI Program and STI 
output by recipients/users in industry, government, and academia. Since the 
Langley STI program is an integral part of the Agency's STI program, NASA 
Headquarters has requested that the survey used by the consulting firm include 
questions pertaining to the Agency-wide STI program and output. 

This task involves (1) preliminary telephone interviewing of NASA STI users 
to supply both content and direction for a closed-ended questionnaire, (2) con- 
structing a closed-ended questionnaire to determine the extent to which the pro- 
gram is meeting the needs of Industrial and academic users of NASA/Langley STI, 
(3) tabulating and analyzing the responses to the questionnaire, and (4) pre- 
senting the findings of the questionnaire in a final report. 

The outcome of Phase IV will be an assessment of Langley's and the Agency's 
programs for meeting the needs of non-NASA users of NASA STI products, services, 
and outputs. 

Phase V; Bibliography 

Phase V of the review and evaluation project requires the development 
of a selected, annotated bibliography of literature citations on the topic 
of the design and evaluation of a scientific and technical information system. 


Phase VI; The NASA Formal Report 

Part I ; Effectiveness of the NASA Formal Report 

Part I of the review and evaluation project requires a comprehensive eval- 
uation of the NASA formal report as an effective medium for transmitting scien- 
tific and technical information. This task involves (1) developing criteria for 
the structure and use of the various report elements, (2) analyzing the 
relationship of those parts within the total report context, and (3) examining 
the overlapping areas of verbal and graphic presentation to determine the 
validity of the present format and/or possible modification. 

The outcome or stated purpose of this evaluation will be the establishment 
of benchmarks by which the NASA report can be evaluated. 


Part II ; Quantitative and Qualitative Criteria for Evaluation 
(Bibliographyr Index, and Tables) 

Part II of the review and evaluation project requires a theoretical and 
analytical review of the formal report as a medium for information transmittal. 

This task includes (1) obtaining, through a manual and computer search, an 
exhaustive bibliography of literature and (2) describing in quantitative terms 
the usage of report components in the report environment. The bibliography will 
contain (1 ) an index of reports produced by government, colleges, and private 
enterprise (acquired during prior research) ; (2) literature which describes the 
usage of components in the scientific/technical report; and (3) literature which 
pertains to the evaluation of these communications elements in the scientific 
report. 

The outcome of the review process will be the development of criteria for 
efficient report organization. 


Part III ; A Review Assessment and Recommendations 

Part III of the review and evaluation project requires an assessment of the 
overall report organization, the component parts of the report, and the rela- 
tionship of those parts within the total report context. This task includes 
(1) contrasting other industry and agency reports (illustrated in prior research) 
with the NASA report, (2) determining which evaluative criteria can be applied to 
the formal evaluation and possible modification of the NASA Langley technical 
report format, (3) establishing a methodology for evaluating the NASA report 
format, (4) outlining a sequence for the component parts and spelling out what 
each should include, and (5) preparing and presenting a final report. 

The outcome of this {diase will be a suggested outline for a sequence and 
hierarchy of parts for specific users and a series of criteria for graphic and 
verbal elements. 



COSTS 


LaRC 

Obligated for; 

Phase I - Knowledge and Attitudes Survey, Langley Research Personnel 
Phase II - Audit of Publication Processes 

Phase III - Audit of the Report and Manuscript Control Office (RAMCO) 
Phase IV - Knowledge and Attitudes Survey, Industrial and Academic 
Personnel 

Phase V - Annotated Bibliography 


Headquarters 

Obligated for: 

Phase VI - The NASA Formal Report 


REPORTING 

The project will be documented in a final summary report. The report 
will be divided into sections containing a review of related research; presen- 
tation and analysis of the data; and summary, findings, conclusions, and recom- 
mendations. Where possible, phases of the project will be presented in individual 
articles. A selected, annotated bibliography on the design and evaluation 
of STI systems will be prepared and published as a NASA Quick-Release Technical 
Memorandum. 



SCHEDULES - PHASES 


Phase/Title 

Feb 

Mar 

Apr 

May 

Jun 

Jul 

Aug 

Sep 

Oct 

Nov 

Dec 

Jan 

Phase I 

Knowledge and Attitudes 
Survey, Langley Research 
Personnel 


1 

■ 

■ 

■ 

■ 

■ 

1 

1 

1 

1 

1 

1 

■ 

■ 

■ 

■ 

■ 

Phase II 

Systems Analysis: Audit 

of Publication Process 

1 

1 

1 

■ 

■ 

■ 

■ 

■ 

■ 

■ 

1 

1 

1 

■ 

■ 

■ 

■ 

1 

■ 

Phase III 

Systems Analysis: Audit 

of the Report and 
Manuscript Control 
Office (RAMOO) 


1 

1 

1 

1 

1 

1 

1 

■ 

■ 

■ 

■ 

I 

■ 

1 

1 

Phase IV 

Knowledge and Attitudes 
Survey, Industrial and 
Academic Personnel 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 


Phase V 

k 











9 

Annotated Bibliography 

1 











9 

Phase VI 

1 

1 

1 

1 

1 

1 

1 

■ 

■ 


H 

g 

The NASA Formal Report 

■ 

■ 

■ 

■ 

9 

Part I: Effectiveness 

of the NASA Formal 
Report 


1 

1 

1 

1 

1 

1 

■ 

■ 

■ 

1 

1 

1 

■ 

1 

Part II: Quantitative 

and Qualitative Criteria 
for Evaluation (Biblio- 
graphy, Index, and Tables) 


1 

1 

1 

1 

1 

1 

1 

1 

■ 

■ 

1 

1 

Part III: A Review 

Assessment and 
Recommendations 


1 

1 

1 

1 

1 

1 

1 

■ 

■ 

■ 

■ 

I 

■ 

■ 

■ 
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